When Open Source Goes Horribly Wrong: Cautionary Tales in Web Development

When Open Source Goes Horribly Wrong: Cautionary Tales in Web Development

By Dr. Shallon Brown

Saturday, April 4th, I virtually spoke during an Enterprise session at the MidWest PHP Conference.

I spoke on the concept of how open source development, for all its benefits, can frequently end up deviating from the original vision of an organization – ending in a project that looks vastly different from its project inception counterpart. But there is a striking reality I encountered while doing research in preparation for this talk that helped shed light on the underlying causes for this type of project failure.

The Honeymoon Phase

When organizations begin to envision life without costly licenses on proprietary systems (which often means gaining back millions of dollars in revenue), they often have the misled notion that this freedom will come with the same general workflow and nuances as with their proprietary software. They also usually believe they gain added benefits, such as:

  • Better return on investment in digital presence
  • Competitive edge
  • Continuously expansive technology
  • Alignment with digital strategy
  • Financial benefits
  • Increased interaction with the desired constituency
  • Easier and cheaper maintenance cycles
  • Freedom from expensive proprietary licenses

These notions aren’t inherently wrong – they actually dovetail with the general progression of industry towards long-term success in organizational management. However, open source is a different beast than many organizations that are accustomed to proprietary systems are used to.

Reality Hits Home Hard

To my surprise, in researching for my presentation, I discovered some statistics that left a real impact on me.

  • 46% of all open source projects fail within the first year.*
  • More than 31% of all web projects that are started are canceled before they make it to launch.*
  • Companies are estimated to invest between $50 to $150 billion per year on failed web projects.*
  • Tech has the highest turnover rate of any business sector at 13.2% (even higher than retail at 13.0%).*
  • The average tenure for engineers at large IT companies (e.g. Google) is 1.1 years.*
  • The biggest reason for web project failure is changing and/or poorly documented requirements.

Yes, that’s right – the IT industry has a turnover rate higher than the retail, food, and restaurant industries, with some of the biggest players in the game struggling to keep people for even a full year. In addition, on the whole, organizations have only about a 50/50 chance of getting a project live. In delving deeper into these numbers, I reflected on my own personal experiences, and what might be the central source of such a stark contrast to the vision of the original goals.

Commonalities in Failed Web Projects

Over the course of my 20+ years in the software industry, I’ve seen hundreds (if not thousands) of web projects that span the gamut of business models – be it government, private sector, or non-profit. Over time, it becomes easier to identify the patterns of what makes for both successful and failed projects.

For a failed project, while many of them do have unique nuances that aggravate the situation, the most common issues in my experience are:

  • Mixing Engineers of Different Specializations Without Universal Training
    While it is alright (even advisable) to cross-train engineers, you need to have at least 1-2 lead engineers who are already seasoned with the project’s specific open source tool set. Without this, it can be difficult to ensure compliance with the standards of said tool, and encourage issues to develop. Even in those cases, training is advisable to bring your engineers up to speed faster, and will ultimately lead to a more sound product the first time around.
  • Lack of Proper Architectural Planning or Preliminary Assessment
    Even with simple brochure-style websites, without proper planning, you are effectively building the car while driving it down the road at the same time.

  • Lack of Risk Assessment
    In the land of open source, risk assessment takes on a different workflow. This is not just an analysis of requirements or functionality pitfalls – it is also an assessment of the implementation parts. For example, in the case of Drupal, your area of lowest risk is unlikely to be Drupal, because it is the most tried-and-true, highly used element of the Drupal community. However, relying on modules that have only a few dozen downloads worldwide for major functionality is high-risk, and should that module fail to work properly, you may need to custom build that functionality from scratch (costing significantly more time and money). As such, adapt risk assessment to your open source model.

  • No Proper Course Correction Early On
    If you are going significantly over budget early on, evaluate where scope creep is happening based on the original requirements, estimates, and discovery process – then realign accordingly. Do not keep burning through cash, or you will inevitably get to a place where there is no more room for the most critical requirements to be built, which defines the heart of project failure.

  • Overestimating the Reliability of Facets of an Open Source Model
    This all started about roughly 50 years ago, with just some huge mainframe machines in a hot room, and a few handfuls of people who knew how to use them. We have since evolved into a world of cloud environments, open source, and other software, many of which are from the open source community. With so many moving parts, there can be a large degree of variability in software testing. As such, do not overestimate the reliability of any one part. Pad your budgets so there is room for hurdles and setbacks, or be prepared for a lot of uncomfortable conversations with your clients as a result.

  • Inadequate Scope Tracking for Open Source
    Open source isn’t always as simple as “install X and setup Y.” Remember to include all the nuances associated with server infrastructure management, local environment setup, etc.

    In addition, for every piece of added scope comes the added overhead with managing said scope. Be sure your hourly rate or budget factors into these critical elements, or it will eat into your developer’s time to build and test the functionality. And of course, pad for setbacks. Very few projects implement fully without at least a few.

  • Requirements Sustainability Issues
    In all of the projects I’ve been through, never once has one concluded without at least some amount of scope increase request throughout the discovery or development lifecycle. Stakeholders will inevitably think of new requests as you talk your way through the nuances of the functionality. Be sure to track and compare carefully.

    Equally important is the detail level of your requirements. For example, creating a select/dropdown list is a simple requirement. But creating a dropdown that then triggers a separate dropdown to update its content in response? That’s a very different (and much more complex) request.

The Moral to the Story

So how does your organization’s next major web project avoid ending up a scary statistic? How can you ensure that your transition or upgrade to an open source platform is less likely to run wild? Here is some sage wisdom based on my experience:

  • Listen to your engineers! They are the heart of your engine room.
  • Building a house without a blueprint is a bad idea… So is building a large scale web application without one.
  • Open source can mean less reliable parts to the system. Plan accordingly, especially when using newer releases.
  • Open source may or may not always be the cheapest option in the end. Choose your stack wisely.
  • Use the right tools for the job, and avoid selecting simply because you are used to a particular stack or CMS.
  • Invest in training, even with an experienced engineering team.
  • Agile isn’t everything – consider hybrid or other models.
  • All software has bugs. Do NOT remediate QA until perfection. You will overrun your budget most of the time doing so.
  • Developer burnout is real, and turnover is considerably more costly to an organization due to supply and demand constraints in our industry. Positions can easily take 6 months or more to fill in many cases.
  • Encourage co-development with your developers, and encourage asking a lot of questions!
  • Running over budget slightly? Get creative and use it to your advantage!
  • Always do a “lessons learned” session upon project completion. Always.
  • If you aren’t tracking requirements, you aren’t catching all scope increase requests.
  • Adapt and change course if things are going haywire, even if it means drastic technical measures.
  • Soft launch is a great concept.

At North Studio, we consider our engineers and supporting team to be our greatest asset.

Having been in the business for over 22 years, we’ve learned a lot about avoiding burnout, engaging with our engineers to make sure their voices are heard, and building a culture that supports long tenures.

Every single member of our management team has been with the company for 5 years or more, and many of our engineers and supporting staff have 10+ years of service with us, which is incredibly rare in our industry. We are proud and humbled by their hard work and dedication, as without them we could not deliver quality products to our clients.

*Sources:

Forbes: https://www.forbes.com/sites/forbesbusinessdevelopmentcouncil/2018/06/29/the-real-problem-with-tech-professionals-high-turnover/#da7fff242014

Statista: https://www.statista.com/statistics/627648/worldwide-software-developer-survey-project-failure/

The Standish Group: https://www.projectsmart.co.uk/white-papers/chaos-report.pdf

CNBC: https://www.cnbc.com/2019/12/03/a-tech-firm-tried-it-all-to-stop-turnover-only-one-thing-worked.html

Work with Us

Your organization can work with some of the most accomplished and accredited specialists in the industry to plan for your next major open source project, and avoid the major pitfalls described herein.

If you would like to learn more about how we can help your organization, we offer a variety of CTO consulting services. We would love to discuss how to make your internal work culture sustainable.

Please reach out to us and mention this blog, and we will provide a complimentary audit review of your system with the purchase of any contract

Meet the author
Dr. Shallon Brown