Private vs Public Repositories in the Enterprise (Enterprise Open Source Part 2)

Why would anyone want to have a private repository (a-la bitbucket), and not make their code available to anyone to see/enhance/critique (a-la GitHub)?

In the public internet, the reasons are legion: plenty of folks are learning and need somewhere safer to put the output of their mediocre, laughable scratchings than on their local disk. Or they foolishly believe that keeping their code secret is a sustainable commercial advantage.

In enterprises, the reasons are more nuanced: you are hired as an (expensive) professional developer, and if you publish your code for others to see (*internally*), folks may question your expertise if they think the code is shoddy or poorly structured. Or you may personally be unhappy with the code, but nevertheless the code is being used to support a valid business need or purpose. etc, etc.

In an enterprise, private repositories actually represent a risk to the firm: if you are maintaining code that is personal to you (or to a small team), then you are providing a service, but not delivering a product. In other words, the value is in you as a person (or the team) and not in the technology you produce as such. There are many examples of this in organisations with a key-man dependency that cannot be broken without paying a king’s ransom. The same theory applies to vendors who maintain proprietary code: make no mistake, they are providing a service, not delivering a product. And so you have a fixed dependency on the service provider, which is an on-going cost and risk, depending on the ‘stability’ of the service provider.

For many organisations, this works fine: banks have been using this model for years, both for traders and for technologists. But it is unsustainable: when people feel their efforts to maintain their service is not being rewarded, they will leave, and then you have an impaired service which could take some time to return to previous levels (if ever).

It should also be noted that in a service-based technology environment, in the event of the demise of a firm (i.e., the people are all fired), the value of the remaining (software) assets are significantly diminished if all the code is tied up in private repositories that have never seen the light of day. So from a ‘living will’ perspective, there is a lot of value in open source.

I believe an enterprise can only move to a product-based environment, where there is sustainable value in the technology itself, when source code is made open (internally), with all the cultural implications of this.

So when would private repositories in an enterprise environment be acceptable? Well, in principle anyone can keep a private repository on a local protected shared drive on a corporate network. But ignoring that, (and apart from code that is genuinely commercially sensitive) firms may explicitly state that private repositories can be used for learning or otherwise developing skills that are not intended for commercial use by the firm. So if you leave, the firm is not exposed.

The rule of thumb should be that all productionised code that is not classified as Confidential should be made (internally) open. And a strict governance process should be in place to ensure that Confidential code is really confidential – i.e., that there is a legitimate business reason why the business users would not want anyone but their own team to see the code, and that only the sensitive code is classified as such, and not (for example) the entire application on the basis it contains a single module with sensitive code. This will be a business decision, and will reflect how much any given business unit in an organisation feel they are part of a wider concern, or just out for their own PnL. 

My bet is that in successful enterprises in the future, business units will be comfortable for other business units to see and leverage their code. But for many organisations today, such collaboration is a major cultural challenge, which IT organisations should help change.

Enterprise Open Source

I recently came across this article, advocating the use of open source within organisations:

Why Your Company Needs To Write More Open Source Software

There are many reasons why going open-source is good for organisations, and almost as many reasons why its not likely to happen any time soon. However, an interim ‘enterprise open source’ approach may be useful – enterprise software written using open-source governance and change control principles, but where the open-ness is restricted to enterprise use.

Why would enterprises want to do this? 

Simply put, technology reuse across enterprises is going to be very, very difficult (if not impossible) to develop in an economically viable way without it. And there are many other benefits, such as giving developers stewardship of business software, encouraging developers to build software that they know will have other eyes reviewing it, improving build/compile automation processes, improving deployment processes, reducing key man dependencies, reducing vendor lock-in, reducing the need for formal code reviews, providing a longer-term ROI for investment in software, etc.

So while not everything in an Enterprise Open Source effort could (or should) be reused, the mere act of building software in this way will improve how software development is perceived and treated in firms that are technology enabled, but are not technology vendors.

Many developers are reluctant to build software that can be exposed to others for critique – this is natural when developers are forced in many circumstances to cut corners to meet project deliverables, even at the expense of technical debt. It’s time to put the technical debt in the spotlight: maybe you don’t have the time to fix it, because it doesn’t need to be fixed for your project. But someone else may think its worth fixing for their project, vs the cost of developing something entirely new.

From an audit and control perspective, I suspect that in well written code, there should be very little sensitive information in code that cannot be shared at the enterprise level: most likely, any such sensitive information should be captured in its own repository, with appropriate controls to restrict access – i.e., a separation of concerns, architecturally.

I believe a top-down sponsored approach to enterprise open source is necessary to allow the development of truly reusable services (encapsulating data and function in one or more modules), modules (encapsulating function only across multiple packages) and packages (encapsulating functional building blocks for modules in the form of JARs etc).

Without it, effective technology reuse will rely only on top-down governance – a level of control and oversight that is expensive to provide and is mostly ineffective, especially for larger organisations. Rather, developers need to be recognised for the code they steward,  and the code they reuse/contribute to, and this combined with a top-down reuse strategy should yield the best outcome.

Would any CTO stick their neck out and advocate this approach?

Note – the above is separate from the issue of whether firms that use open source should contribute back to the community. In particular Amazon has drawn some attention regarding its official policy relating to employees contributions to the open source community. This may be the subject of a future blog post..