Software development is one of those fields where challenging projects, resets and failures are a part of life. This raises the obvious question of why can’t organisations get software projects to work.
According to research, more than half of projects are ‘challenged’, meaning they have overrun costs, been delayed, or failed to deliver all the features. On top of that, around 16% are entirely abandoned, so why?
I regularly see bid packs organisations produce for consulting firms to respond to, and they’re massive. They include pages and pages of details that can be a good sign of danger to come.
The bid pack red flag
The easiest way to spot a project heading for trouble is to look at the bid pack. When an organisation what’s to outsource a project and they’re not sure who to give it to, it will produce a bid pack that explains what they want and why.
The biggest problem with projects at this stage is that they should not really know what they need. When they have a comprehensive bid pack, it means there has been a lot of work already. It often includes requirements that are numbered and set out in a table. It might include the architecture and a list of technologies they want to use. In some large projects, it will break down the project into ‘lots’ and the bid pack will explain how this ‘lot’ will interact with all the other ‘lots’, often with some complex RACI and governance structure.
The reason this is such a big issue is, where has this information come from?
The source of the bid pack
When there is much information in the bid pack it means some work has taken place and decisions have been made. That is not a problem in itself, but it does raise risks.
If work has taken place then who did it and why can they not continue with the project? Either they have not done a good job and need to replaced or they don’t the skills and people to take it all the way. In either case, should the work be relied upon? If the work is poor then no, if the company that produced it lacks resources then it should be treated with caution.
The first part of any project should be some user research. That will tell you if the project has a market and if it should be pursued. This might be done internally or given to a user research company. This is vital information because it informs the company if it is worth hiring a development company to take it forward. Lots of projects end here, and that’s ok. If the project has no market, then it’s best to stop it before you spend too much, but if the research outcome is positive, then it’s worth pursuing. This is the time to get the bid pack together and hire a development company to lead. They will need to do more research, complete the product design, build it and launch it. You will only have a vague understanding of the product and how it will work, but that’s to be expected. The understanding will come from iteration and testing different approaches to the problem.
The detailed bid pack trap
The problems come when more work occurs before the development company is hired. Maybe some UX design work takes place or some more detailed requirements are gathered and documented. Worst of all, some architects have designed the system requirements, created a list of Non-functional requirements and selected the technology.
If the creators of this document cannot take it forward then are they qualified to provide this input in the first place?
Now your bid pack has become a set of constraints. You are telling the development company what the product will do, how it will work and how it will look. You are not asking for a partner to innovate with you. Instead, you are hiring a set of typists to create the code for your product.
Here is the problem, what happens when things go wrong? That design might have problems that are difficult to implement or need further iteration. The team now building it are not the ones who did the research to develop the idea. Are you asking them to continue the research and testing? If you are, then you will probably find the original research was a waste because the new team will repeat most of it. If not, then you are building a product that might not have a market once it actually gets released.
What about the technical design? If a team of architects have come up with the design, then the development team are just building it. You have hired a team of expensive developers who know exactly what they are doing and then telling them how to do their job. The developers will have new ideas about implementing the product based on their real-world experience. They are best placed to design the system, rather than an architecture team who are not responsible for actually building it.
In all these cases, the best possible outcome is that the money spent on research and architecture is wasted. The development team will review it but ultimately take the product in their own direction using the expertise that you are paying for. The worst case is that the additional work is used to box in the approach of the development team. Now they are just doing what you asked for and now taking ownership. They will give you want you asked for but not what you need. They will know it’s not going to work, but they’ll get paid the same.
Motivated developers
Small independent teams deliver the best products. The key is to keep the team small and focus on delivery. Good engineers want to be presented with problems to solve. They want to know what the end goal is and why it is important. They will have years of experience in similar projects that you can use to help you create a great product. If you give them a narrow role, you will get a narrow outcome, and the developers are just expensive typists.
There is nothing fundamentally different about consultant teams. They are a group of individuals who have been hired for the project. Often they will not know about all the details of the bid, so don’t assume the team working on the project will have seen that bid pack. A small team doing their research and analysis will understand the project and be motivated to make it great.
The right approach
This is the best way to approach a project.
- Do some research. This could be internal, phoning a few customers or getting a research company to do some work. Try not to spend too much, but you want to be able to look at yourself in the mirror when you say this idea is worth spending money on.
- Stop. Start thinking about a team to work on the project. If you have an internal team, then that’s an option, but if you need to go external, then start thinking about what you need. You need this team to do detailed research and design a product to meet the user’s needs. They will develop it and run it. They are the ones who will do the work, and it’s all on them.
This sounds simple, but it’s not. Handing over control to a team is not a leap of faith but a managed process. You have confidence in the idea, and the team you are hiring have experience of similar projects in the past. You will want them to explain their vision and approach to you until you feel confident. You should expect to them to show you their data that justifies the approach they are recommending. You should be challenging the roles they need, the timescales and the money, but you also need to listen to them. The relationship needs to be one of trust but not blind trust.
Letting a team run their own process, do their own research and make decisions leaves them empowered to make it successful. There is nowhere to hide if a problem arises, but there is also the problem-solving and leadership that will drive good engineers.
What about big projects?
Sometimes you will have a big project. You need lots of teams, and how do you make them all work together? Do you then need more work before asking for companies to bid? No!
Big projects fail because there is often a layer of management that appears, making the project look harder than it is. You will need some form of project oversight when there are multiple teams, but keep it as light as possible. You don’t want to dictate technologies, but you could say something like ‘the code must be stored in Git’ or ‘it must be deployed to a Kubernetes Cluster’. These wide guidelines are reasonable since they keep some consistency, but they don’t limit teams to specific tools.
You can also split up the work into separate units. A team can work on a product, and another team can work on something else. They need to talk to each other and share ideas, but they should be independent. You might need to share research or design ideas for consistency but let the teams work that out.
The same principles apply, empower a team to deliver something and let them get on with it. As soon as you add planning managers or delivery boards, then the teams have to grow to provide the reports, making the whole thing harder. Often this will be detailed in a bid pack before the teams have even been identified. How could anyone know what they will need?
Let the teams talk to each other, and a process will develop organically.
Conclusion
This has been a rant, but it is important to say. A great idea can be ruined by a project with layers of teams all doing little bits. Get a small team together and give them the freedom to get work done, you will be amazed at what they can do.
It can be tempting to look at the bid pack that has 100 pages, lots of diagrams and tables, plus attachments and think that it represents much useful work. The problem is that it doesn’t. The teams bidding for it will skim it and try to look like they’ve read it, and then they’ll never look at it again. A large bid pack is a waste, think about the elevator pitch.
If you cannot explain to consulting firm what you need in a few sentences, then dozens of pages will not help. All you need to do is find someone who understands the vision. Let them fill in the details, it’s their job.