Do you know how to use regex with C++? Regex had been added in the STL library since C++11! If you exploit "sed" in bash or code in Python, you are certainly already familiar with regex! The regular expressions are able to detect easily a specific pattern in any sequences. During
Software development, facts and quality: A lack of proof?
I think that Software Development and Quality is a fascinating subject. It concerns the analysis of tools and practices which improve productivity. This week, I checked an article from Greg Wilson about Software Development and its comparison with the medicine world.
The article explains the lack of evidence base in the Software development world. When we need to prove something in the medical world, we use studies. In the Software world, there are plain of “headlines”, "punchlines" and "tags" without any proof:
- DRY: Don’t Repeat Yourself.
- KISS: Keep It Simple, Stupid
- YAGNI: You Aren’t Gonna Need It.
- And others
Those claims were emitted by the most brilliant guys in our industry, from merited books or academic journals. However, they are not accompanied by any citation or studies.
In a medical context, Those facts are “Not proven” and would not be “applied” to concrete treatment. Some developers use those beliefs on not proven facts: The base is fragile and could be destroyed by a study.
Of course, these facts are “intuitive” and looks like a great way to improve the code base. In my case, I believe that SOLID principles, DRY and others are a significant “landmark” to drive my coding style. However, they are not linked with proved data. Moreover, some of them are misquoted, as explained in the article by Greg Wilson on quote “The best programmers are up to 28 times more productive than the worst”.
One example of proven/not proven data: Test-Driven Development.
Featured by some developers as the “magical pill”, TDD is also a practice with unwillingness for many reasons:
- TDD requires the skill to write “Good” and “Short” tests.
- Without proof, nobody wants to put effort into it. Do TDD worth it?
Fortunately, TDD is the subject of many recent studies. They are proof of the benefits and disadvantages of TDD. Without using “tags”, they are a weapon to convince developers and managers about “the goods and evils “of TDD.
For those who do not know what TDD is, I will simplify the definition. It is the process to write the test before the development of a feature. Those short tests are in the “Green Red Refactor” cycle:
- Write “Failing” test – RED - FAILED
- Write minimal code to pass those tests – GREEN - SUCCEED
- Refactor the code – REFACTOR
- Re-run the test to pass it GREEN, and restart the process with another minimal feature.
Thanks to this process, TDD allows tests to guide the implementation to maintain a supposed “higher-quality” code. Even if it is not yet proven, multiple studies are in progress about the benefits and disadvantages of TDD!
Some of them claim that TDD improves productivity, other that it decreases productivity. Sometimes it depends on the industry. Maybe, it depends on the project size or other!
But overall, in the industry world, current studies show that TDD improves quality, bug count and improves productivity in the long term (when it decreases short time). This kind of evidence is a weapon to convince you to embrace or not the TDD development style.
As you see, the Software world is full of claimed facts. Some developers will feature those claims as a “quality elixir”. I do not think it is an excellent way to improve the software world. Some leaders force the introductions of those facts, which add tensions, without guarantee of quality.
I keep in mind those headlines to guide my own “coding style”, but not to force a complete team without agreements.
Did you already encounter this issue?
During your work, your boss asks you to add some cool feature to a car app. You are ready to analyze the state of art, you open the source code and… Damned! What happened here? What is this home-made XML parser? Why are we using Unix sockets here?
You continue your investigation and find tab/spaces in the same file. Bad indentation and then, the usage of multiple deprecated standards. No doubt, you are analyzing an old code base not standardized and without coding guidelines.
The issue of code without coding guidelines.
During my career, on old code base, I encounter some strange patterns: Homemade parsers, re implementation of std::string, multiple usage of C style string in no critical parts… In huge code bases, the number of developers is proportional to the risk of multiple tools integration.
The issue of code without coding guidelines that it increases the general complexity:
- More libraries to verify during upgrading.
- Questionable code quality.
- Code harder to read.
- Losses of knowledge
- Multiple documentations to read.
However, each teams or individuals want to integrate their own tools. It is the goal of the leaders to control those tools and limit the multiplication of those libraries. I will feature here some simple solution to set standards in your code base.
An easy solution: Communication and coding guidelines usage
It is the obvious one, but also the harder to implement!
The first goal is always to communicate with the team about your objectives.
Before each library integration, you should communicate about your intentions. Debate with your colleagues and select a single tool to support this JSON parsing!
The second step is to adopt coding guidelines. A good code guideline consists to give a set of rules for using a code language in a particular environment. A coding standard is always better than no coding standard. Of course, a good standard should be written by someone with solid knowledge about the concerned language.
Therefore, my recommendation is to use an existing code guideline. You have some great examples in C++:
The usage of those guidelines will save you lot of time in the future:
- tab/space usages.
- Unitary test emplacement.
- Guidelines concerning usage of floats/double/threads.
- Easy support: The other team members use the same standards!
- Better code analysis.
- Consistency in your code.
However, be careful: Some software engineer will reject your coding standards. In majority, those are developers with obsolete skills. They will try to ignore your standards and you should defend the value of code standardization.
The goal here is simple: Have a simple set of rules that help developers to edit any part of the code. Important in big code bases!
Do not hesitate to check this quite interesting article from isocpp about coding guidelines.
Automation and tools.
You defined a coding guideline in your team. Congratulation, you completed the first task!
However, some developers still not respect those standards.
After months, the standards are forgiven. It is important to use automatized tools to support your code standards and force this new policy.
To help the adoption of your guidelines, you should define tools and automatic verification.
The easier to configure is your IDE! There are multiple tools to define code rules and helps for automatic verification.
For instance, you can use the Code Spell Checker in Visual Studio Code.
In eclipse, you have the Static Analysis configuration: StaticAnalysis
For another cases, you can also use some static code analysis tools. Such as CPPCheck.
And here the more important: C++ standards are evolving. Do not hesitate to edit your guidelines with the evolution of C++!
Photo by Vamsi Konduri on Unsplash
Today, I review with you some interesting blog posts!
Post update: Regular Expressions in C++.
First, I updated my old post about regular expressions, do not hesitate to review it! You have now new code blocks and extra tests!
Why rewriting a code from scratch can be a terrible error?
This week, during my research, I found an old blog post from Joel Spolsky named “Things You Should Never Do, Part I”.
The post is 20 years old but is still relevant today! It explains why rewriting a code from scratch can be a terrible error for a company.
For me, rewriting a code from scratch is a double or nothing bet. It can be productive in some special cases (Very old code base, new standards, revolutionary technologies, etc.…). However, in majority, it is inefficient and counterproductive for commercial usage. We can dispatch the common issue in two parts: Architectural and Performance.
Concerning architectural issues, they can be fixed smoothly. Time by time, I can modify an interface. This other time, I refactor a class during a development. And again, let us rewrite this single part which is slow!
Trust me, I applied this method during years on an old project. The result today is a code modernized and easier to understand compared to few years ago. First, modernize build tools. Then, refactor the code iteration after iteration is pretty efficient.
Concerning the optimization... Do you have a code which is not performant (slow, memory leaks, etc…)? Let identify the part and do not rewrite the entire code! A simple modification of the concerned part is maybe better for the commercial usage. Your code is immediately deliverable and productive.
Of course, some errors are hard to fix (Threading, shared pointer...). But I doubt it is a reason to throw it in the trash bin.
The code I am still ashamed of.
This post is a particular one. Have you ever been ashamed of your own code? In an ethical way?
I already encountered some questionable decision during my career.
You may know this well-known post from Bill Sourour: The code I’m still ashamed of.
The question of ethical in software is still important: There is a chance that during your career, someone will ask you to code something a little deceitful. This experience from Bill Sourour is touching and terrible. It reveals how you can code something unethical and why it is important to say “No”.
In my personal experience, I already encountered this. Fortunately, leaders were listening, and it was resolved quickly (Just say no and explain why it could cause problems). But I supposed it is not the case for everyone. Moreover, saying "No" is far more easily said than done. You could have terrible leaders which refuses to listen your concerns and stay the course.
Do you train your programming skills? Do you follow any programs to keep your programming skills up to date?
It was not the case for me one year ago. Currently, I am maintaining my level and learning some new programming languages! My training is based on three axes: Book reading, learning new languages, and skills maintenance.
The first pillar of programming skills training: Book reading.
The first part of my training is dedicated to book reading. Each day, I take about 15-30min of my time to read books.
At first, I read some “no technical books”. Like personal development in the programming world. I can recommend you some books like this one:
Here is a little review of "Pragmatic Programmer". Indeed, This is one of my favorite "programming" books". Written by Andy Hunt and Dave Thomas, two professional software developers, this book reviews different situations in the Software industry. Furthermore, I felt that this book was really easy to read: Few codes
In all cases, this training has three goals: To learn new skills, new practicals and the evolution of the programming world. When I started to read books, I felt that my Software skills had been improved. Among all my recommendations here, it is the best one!
Learn a new language per year.
The second part of my training is learning a new programming language. The programming world is in constant evolution. A few years ago, Typescript didn’t exist. Docker is recent and now Virtual Reality and Artificial Intelligence are coming up!
As Software developers, we should learn new competencies constantly. Personally, I am learning one programming language per year.
To learn a new programming language, you don’t need to learn all specificity. My program to be proficient is simply to start by reading a book/following a course. Then, I start a personal project to practice my new language.
You don’t need to know a new programming language at 100%. Only 20-40% of the language is sufficient to be proficient!
Maintain your programming skills: The code Kata.
There are many programming languages. However, in a single job, you can exercise only a few programming languages. Moreover, “coding” is only a few parts of our job. You participate to meetings, maintain designs, review strategies...
Your programming skills are rotting. Specially the ones you don't use. We should maintain them to be proficient at any time.
To maintain our programming skills, we should train it! Personally, I follow a great method featured by Dave Thomas: the Code Kata! Later, Robert C. Martin described the concept in his book “The Clean Coder”.
In martial art, a Kata is a choreographed movement that simulates one side of combat. Converted to the programming world, a Kata is a precise set of movements to resolve a simple programming problem. It can be “How to convert a string to uppercase” or “Create a simple chess game”.
Choose your programming language, and repeat the exercise again, again, and again! Practicing a set of Kata is a great way to maintain your programming skills. Personally, I follow some websites to train my programming skills in multiple languages, like C, C++, Rust, or Python. Here two of them:
Those websites propose to you a set of simple exercises to train your coding skill. Enjoy!
During this week, I worked on an old system that still uses a C++98 compiler (apparently, popular in some industry like spaces, airport…).
During a code review, a colleague tried to share a pointer between several classes. Unfortunately, without smart pointers, it is hard to protect this single pointer against race conditions.
After reviewing several solutions, equivalents of shared pointers, etc… I asked myself “Am I going on the correct way? Why are we protecting this one only for performance matter?”. Indeed, the simplest way was to copy this data, and I suggested to go with this simple solution.
Even if the performances are not optimized, the application is still efficient. It is why I am writing about this subject right now. What is more important in a code? Code performance or simplicity?
Optimization is not the goal.
Some developers, especially the fresh ones, think that the optimization and performance are the goal of a software. No, the primary goal of software is to be efficient!
I encountered a strange situation about an embedded system software: I saw some old code using a single “short” to store a set of Boolean. Thanks to a binary operation, we can get the values and then treat them:
C++ - A strange optimization
void connect_usb1(short system_status) { if (system_status & 0x4) { power_on_usb(); } }
Pretty hard to read… For weeks I reviewed those kinds of code developed in the name of optimization. Every developer tried to read this, treat those strange set of bits, but every time we got difficulties. One day, I reviewed this code and with a colleague, we remove this kind of optimization to make the code simpler. I think you can easily understand the goal with this second example:
C++ - Simplicity
void connect_usb1(system_status current_system_status) { if (current_system_status.is_usb1_powered_on) { power_on_usb(); } }
Sure, we are using a bigger data set… But the code became readable, easy to debug code, and easy to understand.
This is why optimization is a trap. The optimization can destroy the readability of your code. More of that, the “optimization” details are… Useless.
The compiler optimizes your code!
Did you see my post about ++i vs i++? If it is the case, you should remember how the compiler optimized my code with i++.
This example is true for any kind of instruction. In reality, the optimization is done in the majority by the compiler without any actions from you.
More of that, our systems today are so powerful that a compiler can extremely optimize your software.
Today, our systems are so powerful that we can sacrifice some performance for simplicity.
Is it useless to optimize my Software?
No. The question Simplicity vs Performance doesn’t have a universal answer. I got several experiences where the optimization was primordial. A great example is when I worked in the car engines. With important real-time restrictions, we optimized perfectly the software by using mapping and fixed points.
Another example when I worked for a hardware logger framework. As the logger was regularly used in this Software, I optimized this one too.
I cannot count the number of projects falling because a database was heavy to execute. Like the simplicity, the lack of optimization can kill a project. Simplicity and Optimization are not enemies, they work together.
You could optimize your code if you are working on a “core concept”, “critical part” or “a piece of code which is regularly called”. Indeed, only 20% of your code is used 80% during run-time. (Some say even 10% of the code used in 90% run-time!). It is those core concepts to optimize! But above all, don’t sacrifice the readability for a simple optimization!
If a colleague says that his solution is 12% more performant. Say no!
We saw the followings:
- Optimization doesn’t mean “efficient” and “simplicity”.
- The compiler optimizes your code.
- The optimization question is valuable for critical parts.
- Only a few parts of your code are regularly used.
If during a code review, a colleague proposes a homemade system, ask if the game is worth the candle. If it reduces the readability, if it doesn’t concern a core concept, say no!
One day, another developer will work on your code. If the code is not simple and easy to read, the risk of regression is higher.
The priority of Software should be efficiency and maintainability. The question of optimization is valuable only about some critical parts.
Is OOP (Object Oriented Programming) dying? Should we stop to use OOP? This question is asked by many programmers. I can't count the number of post about "why OOP are great" or "No stop! It is a trap!". I wanted to contribute to this happy debate!
Why OOP is criticized?
Yesterday, I read this quite interesting blog post written by "Charles Scalfani".
He is an experimented Software Developer who developed with OOP during decades. Here, he focused on the "Three Pillars of the Paradigm": Inheritance, Encapsulation and Polymorphism.
Of course, OOP is criticized because it is a popular programming style! During this post, I will review some arguments gave by Charles Scalfani and offer my opinion. Here we go!
First, the Inheritance
Inheritance is the biggest benefit of the Oriented Object Paradigm. It sounds perfect for reusing and avoid duplicate codes.
Looks great ! Let's use inheritance to guaranties clean code and easy reuse!
...
No, we cannot use inheritance everywhere. Bad usage and mastery of inheritance is WORSE than no inheritance.
Banana Jungle Monkey - The reuse killer in OOP
During my carrier, I worked on an old Software developed in C++ with about 500.000 lines of code. The code was terrible and needed refactoring. During a featured development, we said: "Hey, we could take this class for inheritance, it is perfect!"
OK, let's inherit this class... Hugh... It needs another class... OK, let's take this.
And another class... And two others... Oh damned, we need an entire framework now...
And then, the situation was so complicated that we needed to create a new class. Goodbye reuse, welcome to duplicated code!
I agree with Charles Scalfani about inheritance. Moreover, he gave a great quote from Joe Armstrong, creator of Erlang:
”
The problem with object-oriented languages is they’ve got all this implicit environment that they carry around with them. You wanted a banana but what you got was a gorilla holding the banana and the entire jungle.
Joe Armstrong // creator of Erlang
For a simple compatibility, we got all the environment around this class.
Too many inheritances in a Software can limit reuse. This kind of code can be hard to read.
Limit the inheritance issue in OOP.
Several solutions exist to limit bad inheritance designs in OOP:
- Composition over inheritance. Containing instances of other classes that implement the desired function rather than inheritance. This is the solution against the Diamond Problem in many OOP languages. True for languages which doesn't support multiple inheritance, like Java.
- Develop a class and inheritance hierarchy. External documents which are hard to maintain...
In result, we delegate and limit the usage of inheritance to avoid those kinds of issue.
Unlucky! Inheritance has so many drawbacks...
I featured here only one of the potential issues by using inheritance.
- Bugs because implementation in the based class changed.
- Inheritance of objects which inherits of 6 others... Hard to maintain and read.
- External documentation to maintain a readable scheme.
Inheritance is hard to maintain and can conduct to bad design. To avoid a huge quantity of documents and comments, inheritance should be limited.
Second, the Encapsulation.
In this part, I don't agree with the author.
Encapsulation is great. Structuring data in objects is one of the best benefit of OOP.
Of course, there is the "Reference Problem". But this issue can be resolved by using a deep copy. Look easy to implement!
Passing object by reference can lead terrible after comes in security. Simple discipline and knowledge to limit those kinds of errors.
Third, Polymorphism in OOP.
Another part where I don't agree...
I used polymorphism for reuse code when inheritance fitted well.
It can save a lot of time by using methods still tested and approved.
Fourth? The missing pillar: Abstraction.
Abstraction is the fourth pillar. It is an extension of encapsulation.
OOP manages well the abstraction (hiding the information!). It suits well to reduce complexity by showing only necessary information. It helps to expose only necessary details.
I don't see any important default concerning abstraction...
The four pillars are alive... But...
OOP fails about potential issues leading to heavy design. OOP is not a principle easy to learn, concepts introduced are more complex than functional programming.
Moreover, the code can be heavy by introducing artificial classes... OOP fits well for organization. But this organization can be a trap, leading to the other defaults of OOP:
- Risk of processing time limited.
- Program larges in size.
- Risk of not relevant classes.
- ...
Is OOP dying?
Quick answer: No.
Oriented Object Programming is still a good tool for several reasons:
- Code reusable thanks to inheritance.
- Modularity thanks to encapsulation.
- Flexibility thanks to polymorphism.
- Security thanks to abstraction.
Of course, some tools (like inheritance) should be used with precaution... All along this post, I criticized the "inheritance principle". But inheritance issues are resolved by using great and clean designs.
When issues occur with OOP, it is due to flawed architecture.
Despite this, I found the blog post introduced interesting: Even if it looks exaggerated, it shows why OOP is not perfect.
Yes, OOP is not perfect and is not a solution all kind of problems. Sometimes, it is better to use another programming style:
- Functional programming.
- Logic programming .
- Procedure-based programming.
- etc.
Here is a little review of "The Pragmatic Programmer". Indeed, This is one of my favorite "programming" books".
Written by Andy Hunt and Dave Thomas, two professional software developers, this book reviews different situations in the Software industry. Furthermore, I felt that this book was really easy to read: Few codes and tips available whatever the programming language. In fact, it is a great entry in the Software world for each new comers in the Software industry. If you are a veteran in the industry, I am not sure that you'll learn a lot!
With "The Pragmatic Programmer", you'll review several point of the industry, specially:
- How to fight the technical debt (rot code).
- How to don't repeat yourself.
- Create flexible, dynamic and adaptable code.
- Capture the real requirement.
- The essence of agility.
- And other stuffs...
Despite the dated examples (The first edition was released in 1999!), the different anecdote are still relevant!
From my personal experience, this is my first programming book and the one which motivated me to create this blog. After, I continued to read more and more book and become a better Software Developer. More of that, this book gave to me "words" to several practice I done during my carrier.
Also, it reveals one aspect that I specially liked. The parallel between a Software Developer and a "Gardener". Indeed, as a gardener, a developer must regularly work on old and legacy system. Those legacy system should be maintained regularly to avoid technical debt (and then, keep a minimum of modernity!).
If I should recommend one book for a fresh Software Developer, it would be probably this one. So, don't hesitate to review it!
The Technical Debt (also named "Tech Debt" or "Rot Code") is a concept which reflects a technical solution which doesn't fit anymore the needs.
A Technical Debt can be integrated by multiple causes, deliberate or not:
- Business pressure. The business can consider that a feature is released sooner before the code is completed.
- Code is not maintained (We can name this a code which smells! It is maybe where the "Rot Code" term is coming from).
- Lack of knowledge.
- Specification modified at the last minute.
- Other, other and multiple other cases...
Technical Debt: A parallel with the monetary debt.
Each time a mess enters in code, you integrate in the software a "debt". If you don't re-pay this debt (re-worked), you pay "interest" - it means, development time -.
Here, You can see my (very) simple scheme to illustrate the tech debt matter:
- In green, Project A is a project where the code is regularly maintained. It was designed to be growth.
- In red, Project B is a project elaborate for quick market release.
In result, as you see, After few time (matter of weeks or months), the Project B is paying interest compared to Project A.
This is the debt accumulated.
Advantages and dangers of technical debt.
In order to illustrate advantages and dangers of tech debt, I show to you a simple scenario with Project A and B.
Let's do a parallel with a GPS application:
- The first one, Project A is with a code structure well organized. The project is scalable and the team resolves regularly tech debt issues.
- The second one, Project B is not well organized. The tools are not scalable and the priority is fully dedicated to the feature delivery.
- Project B had been released before Project A. Project B costs certainly less than Project A (15$ vs 20$). During the first year, Project B is a must-have and help many drivers to find their way.
- After 1 years, Project A is released and does exactly the same thing as Project B.
- At last, After 2 years, Project B is forgotten: Project A contains many features compare to the concurency ! It can save the home location, has vocal assistance and can foresee the traffic jam! Meanwhile, Project B is sold at the same price and is unable to overtake Project A.
I think you can make the parallel with multiple projects in the software industry.
Indeed, you got it! Even if Project B was a success story at the beginning, the management didn't dedicate resources to resolve the accumulated debt. In the long term, adding features became painful and the development team was under a permanent pressure. Money is lost, developers and managers was under pressure and the project is disbanded. This is a doomed fate...
Tech debt accumulation
How tech debt can impact a Sofware project
Pros
Cons
Conclusion:
The technical debt accumulation is extremely dangerous in medium and long term.
Unfortunately, even if this concept is well known by developers, managers ignore regularly how many tech debt is accumulated in projects. Indeed, the tech debt is hard to estimate in terms of time and money lost.
Personally, I already joined a new formed team to recover a project with a huge quantity of technical debt. Despite the fact to release quickly, we stayed professional. We proposed for each release a simple refactoring of the code or redesign of messed parts. Some of my previous colleagues are reading my blog, they will recognize! =)
In result, after one year, the project became cleaner and the environment "developer friendly".
The Technical Debt is an extremely large subject and treated in many blog posts and books. Here, I featured to you only a tiny part. I'll certainly review this subject for another blog post (specially to tell you my experience in this project with many tech debt!).
For more information, you can view this presentation from Steve McConnel.
In parallel, tech debt can be caused by design not secured or other messes, don't hesitate to review my blog posts about security or clean code:
Hey, I want to feature you quickly this book (A classic one!).
Clean Code by Robert C.Martin (also known as Uncle Bob). Robert C.Martin is a Software engineer and instructor. He is also known as one of the authors of the famous Agile Manifesto!
I read this book for the first time few month ago. And I'll certainly restart in one year.
Why? Because the examples are timeless and in total abstraction of the language used! It is a great introduction to recognize what is the difference between good code and bad code. Quick example of the principle featured:
- Choose descriptive names for your variables - In one year, you may probably forgotten the meaning of this strange variable named "testTmp" !
- Keep Configurable Data at High Levels - Avoid magic number and simplify the configuration of your variables with default value!
- Avoid dead code (even commented)! - Your source control system is able to remember all the history.
There are many other example and better illustrated than I did above. It also illustrate at the perfection the "Boy Scout Principle".
"Leave your code better than you found it"
“Can you imagine working on a project where the code simply got better as time passed ? Do you believe that any other option is professional? Indeed, isn't continuous improvement an intrinsic part of professionalism?”
- Robert C.Martin - Clean Code
The question of "Why a clean code is important in the industry" and "Is it vital for a business to keep clean code" is a question which deserve several post due to its complexity and multiple opinions.
In all cases, I think it is a great book for every developers which want to improve. So, don't hesitate to acquire this book, it is a must have!
