Nobody likes making time and cost estimates, but they are an important part of business. This is especially difficult for programming because many programming projects are unique.
Many types of non-programming projects are standard. If someone wants a house, an engineer and architect don’t have to completely invent the building technology, they fall back on tried and true methods. Programming projects often deal with new things. This could be standard PC software, software for a phone app, an operating system, drivers, embedded device, electro-mechanical systems or more. Each has their own set of challenges.
The first step is to solve the problem. This solution doesn’t have to be detailed, it just has to be enough to grasp the problem and how it can be resolved. The solution may not work, but that’s a matter of experience. What this first step does it gives you a direction for research. Research is vital for removing unknowns. Unfortunately most clients expect you to swallow this cost.
Unknowns are the largest cause of missed programming deadlines. They come in many forms, from flawed libraries to problems in the operating system or language. Some projects may take five times longer simply for this reason. Experience with the required language or libraries helps, but it’s far better to use a set of trusted libraries and a well polished language. Avoid flavor-of-the-month type frameworks because these cause no end of problems.
The next hurdle involves understanding the skill required to complete a project compared to the skill you have available. For example, not all programmers can write a programming language, so if a solution requires writing a programming language then you may want to choose another solution. Skill difference is a rather strong factor. A complex section could take twenty times longer for someone without sufficient skill. They may have to rewrite it over and over before it works, whereas a more skilled developer could do the same in a single pass. If the section is beyond the skill of a programmer then he simply can’t create it, regardless of time.
Special requirements are the next challenge. Special requirements are requirements that pass well beyond normal assumed limits. Suppose your application has to store twenty terabytes of data per day. Typical databases would struggle with that, so you may have to write a custom database engine. If you’re writing a website and it has to handle ten thousand requests per second then that becomes the pivotal point. Everything must be designed with that in mind.
The largest time factor in a project is overlapping options. Sales or marketing always want to pack the software full of options. They want it because it makes it easier to sell. That’s because other sales types handle purchasing and they believe option rich software is better. In reality, using the fewest possible options creates a far superior software product. In addition to simplifying the interface it simplifies the code. This is due to the combination of options growing exponentially. If software doesn’t correctly handle all possible option combinations, then it’s buggy and becomes a support nightmare. Not all options overlap, so you calculate this based only on overlapping options. For example, if there are three options with four states, and two options with two states, then the total combinations are: 4*4*4*2*2 = 256. Every combination has to be designed, programmed and tested. If there are twice as many overlapping options, that goes from 256 cases to 256*256 = 65,536 cases. Too many overlapping options is the fastest way to kill a project.
Next you need to ensure your programmers are properly trained. Many programmers cut corners. They don’t properly organize or structure their code. They hack in lines of code until it becomes a jumbled mess. The common term for this is technical debt. Rapid advancement can be possible with increased technical debt, but from that point on development will slow as you work through accumulated debt. Like all debt, technical debt has interest. So ignore the illusion of progress and do things right from the start.
Beware 80% solutions. Various approaches cause rapid initial progress. Sometimes it’s tempting to take one of these, to demonstrate progress to stakeholders. The basic idea is that 20% of the code is used 80% of the time. So you write that 20% first and add in the rest later. When this happens, you structure the code around that 20% rather than looking at all cases. This ends up taking 150% more work or more. Sometimes it’s impossible to complete the remaining code, depending on how the program is structured.
Finally solve the problem completely before you start a project. If you can’t solve the problem then don’t even bid on the project. Sometimes you’ll begin a project with an untested idea, without knowing for certain if it will solve the problem. A good example of this is self driving cars. People working on this problem use neural nets, because it looks like it might work and it has a fast initial response. It’s an 80% solution that will probably never reach 100%. A far better approach is an abstracted model algorithm. That would take far longer to initially develop but it would reach a 100% solution far faster.
There are all kinds of pitfalls, but people expect estimates so it’s a required skill. It takes experience and practice, so before each project make an estimate even if it isn’t required. The feedback from this will help you learn the skill.