The Story of Bug Hunting in Production

BUGS COME AND GO. DEVELOPERS STAY!

Production bugs mean you cannot debug line by line; even worse that cannot reproduce them. The other days, I got one of them. We know when it happened by a screenshot reported by the client.

After a day hunting the bug, I finally found out the root cause. Once found out, the solution is a piece of cake. I thought the process I had been through is interesting and might be useful for others.

Here we go – the process of hunting a production bug.

Step 1 – Treasure in the Log

The first thing is to get the log. Find the entries at that specific time. Most of the system will display a user-friendly error message to end users. The real error, usually in the form of Exception, is buried in the log.

System.Transactions.TransactionAbortedException: The transaction has aborted. ---> NHibernate.Exceptions.GenericADOException: could not execute batch command.[SQL: SQL not available] ---> System.Data.SqlClient.SqlException: String or binary data would be truncated.
The statement has been terminated.
   at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
   at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
   at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString)
   at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async, Int32 timeout, Task& task, Boolean asyncWrite, SqlDataReader ds, Boolean describeParameterEncryptionRequest)
   at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, TaskCompletionSource`1 completion, Int32 timeout, Task& task, Boolean asyncWrite)
   at System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(TaskCompletionSource`1 completion, String methodName, Boolean sendToPipe, Int32 timeout, Boolean asyncWrite)
   at System.Data.SqlClient.SqlCommand.ExecuteNonQuery()
   at System.Data.SqlClient.SqlCommandSet.ExecuteNonQuery()
   at NHibernate.AdoNet.SqlClientBatchingBatcher.DoExecuteBatch(IDbCommand ps)
   --- End of inner exception stack trace ---
   at NHibernate.AdoNet.SqlClientBatchingBatcher.DoExecuteBatch(IDbCommand ps)
   at NHibernate.AdoNet.AbstractBatcher.ExecuteBatchWithTiming(IDbCommand ps)
   at NHibernate.AdoNet.AbstractBatcher.ExecuteBatch()
   at NHibernate.AdoNet.AbstractBatcher.PrepareBatchCommand(CommandType type, SqlString sql, SqlType[] parameterTypes)
   at NHibernate.Persister.Entity.AbstractEntityPersister.Update(Object id, Object[] fields, Object[] oldFields, Object rowId, Boolean[] includeProperty, Int32 j, Object oldVersion, Object obj, SqlCommandInfo sql, ISessionImplementor session)
   at NHibernate.Persister.Entity.AbstractEntityPersister.UpdateOrInsert(Object id, Object[] fields, Object[] oldFields, Object rowId, Boolean[] includeProperty, Int32 j, Object oldVersion, Object obj, SqlCommandInfo sql, ISessionImplementor session)
   at NHibernate.Persister.Entity.AbstractEntityPersister.Update(Object id, Object[] fields, Int32[] dirtyFields, Boolean hasDirtyCollection, Object[] oldFields, Object oldVersion, Object obj, Object rowId, ISessionImplementor session)
   at NHibernate.Action.EntityUpdateAction.Execute()
   at NHibernate.Engine.ActionQueue.Execute(IExecutable executable)
   at NHibernate.Engine.ActionQueue.ExecuteActions(IList list)
   at NHibernate.Engine.ActionQueue.ExecuteActions()
   at NHibernate.Event.Default.DefaultFlushEventListener.OnFlush(FlushEvent event)
   at NHibernate.Impl.SessionImpl.Flush()
   at NHibernate.Transaction.AdoNetWithDistributedTransactionFactory.DistributedTransactionContext.System.Transactions.IEnlistmentNotification.Prepare(PreparingEnlistment preparingEnlistment)

Our system leverages NHibernate for Data Access layer. And the exception tells us that there is a piece of string that does not fit into its column max length. It happens when the system commits a transaction.

The question is which table/column? There are many tables involved in a single transaction. This is a complex system.

In the bug’s context, there is very few information involved. I looked at the code that is responsible for the function. Nothing’s special.

Step 2 – Look for Variants

From the customer, I know that some users had the problem. Some have not. Therefore, logged in user plays an important role in the bug.

When looking at any problem, there are always a number of participants, there are input and output. I took a pencil and paper. I jotted down all the participants (actually the C# object/class, such as User, Workflow, …). Just jot them down. Then I drew relationships between them.

Thinking about both cases success and fail, what are the variant factors? Luckily in my case, I can quickly figure out that was the User. And in our system, we care most about 2 pieces of information: Username and Full Name.

This step is important because it will give us a good picture of what might cause the problem.

Step 3 – Look at Data

Luckily that we can access to the production database. Ran some queries. The data emerged an interesting observation:

If the username is 41 characters, it causes the issue. If the username is shorter (at that time I did not notice how many characters the shorter means). I knew that our system keeps the created/edited information. But those columns have more than 50 characters max. It should not be an issue anyway.

Let’s find all the columns having the limit of 41 characters

SELECT TOP 1000 [TABLE_CATALOG]
      ,[TABLE_NAME]
      ,[COLUMN_NAME]
      ,[DATA_TYPE]
      ,[CHARACTER_MAXIMUM_LENGTH]
  FROM [INFORMATION_SCHEMA].[COLUMNS]
  WHERE CHARACTER_MAXIMUM_LENGTH < 41 AND CHARACTER_MAXIMUM_LENGTH > 0 AND DATA_TYPE='nvarchar'

Hurray, the bad boy could not hide anymore. There I had a very small set of columns which might cause the issue. It did not take much time to check each of them. All I had to do is to check in the code which column accepts Username value.

The fix? Well easy. Just need to extend the column max length.

 

Dealing with production bugs is hard. There is very limited information. And for most developers, if they cannot reproduce, debug the bug, they give up. The first and most important thing is to have a right mindset:

There must be something wrong somewhere in the system.

Fix a production bug? Not a problem!

  1. Have the right mindset. Do not assume.
  2. Gather whatever we can. Use whatever we got. Do not complain.
  3. Take a deep breath.
  4. Apply basic logical thinking. The process I share here is one of them.
  5. Hey, need a bit of luck.

Either you like it or not, bugs happen in production. I suggest you embrace them, have fun with them.

BUGS COME AND GO. DEVELOPERS STAY!

Code That Embraces Changes

Developers write code. They write code for specific functionalities, with requirements from clients (customers, bosses, leaders,…). When requirements come, they know what to do; know what code to write. In the world of business applications, the implementation is not a big deal. The story might be different if we write code for embedded systems using C/C++.

However, most of the problems come later, when the first draft of the function has done. The functions have deployed to the customers. They test and realize that they have to change here and there. If I have to pick one constant in the software development, I would pick CHANGE. The clients will change their mind. They know better when they actually see the product. You, as a developer, cannot avoid changes and cannot tell clients stop changing. No. It does not work that way.

Agile emerges with the spirit: Embrace changes. A wonderful mindset shift! We have to adapt that fact. However, the biggest question remains: How do you deal with changes?

How do you deal with changes in code?

I know that there are processes to deal with changes. But, ultimately, it all comes down to the code. You have to change the code, most of the time you have to.

I do not have a single answer or solution to that problem. However, I do have some experience in designing code that gives me the courage, the tool to embrace changes.

Here we go. But first, let’s define a simple feature. We have to work on a concrete example.

Requirement

As a business owner, I want to export orders (that are made today) into CVS file so that I can import into Excel and build some charts.

Assumption: There is a shopping system. They take orders from users. The orders are stored in the database. They are all basic stuff if you have ever worked with a business application system.

First Implementation

Without any hesitation, jump right into the VS2017 and implement the feature. Btw, how hard it is to implement such a simple feature, right?

    public class Order
    {
        public string Customer { get; set; }
        public DateTime OrderedAt { get; set; }
        public decimal TotalCost { get; set; }
        public int NumberOfProducts {get;set;}
    }
    class Program
    {
        static void Main(string[] args)
        {
            var ordersFromDatabase = new List<Order>
            {
                new Order{Customer = "Batman", TotalCost = 100000, OrderedAt = DateTime.Today.AddDays(-3)},
                new Order{Customer = "Superman", TotalCost = 1000, OrderedAt = DateTime.Today.AddDays(-2)},
                new Order{Customer = "Spiderman", TotalCost = 100, OrderedAt = DateTime.Today.AddDays(-1)}
            };
            const string headers = "Customer;Total Cost;Ordered At";
            var data = new List<string>{headers};
            foreach (var order in ordersFromDatabase)
            {
                data.Add(string.Join(";", order.Customer, order.TotalCost, order.OrderedAt));
            }
            var fullPathToCsvFile = @"d:\export\orders.csv";
            File.WriteAllLines(fullPathToCsvFile, data, Encoding.Unicode);
        }
    }

The actual implementation in a real project will be different. To demonstrate the point, I simplify the code.

What do we have above?

  1. An order has 3 information: Customer, Total cost, and Ordered At.
  2. There are 3 orders for Batman, Superman, and Spiderman. Assuming that they come from the database. Batman is a rich man so he ordered a lot 😛
  3. Export to CVS with “;” deliminator.
  4. The file is located at “d:\export\orders.csv” with Unicode encoding.

The code works. Mission accomplished.

Changes Come

Here are some changes that a customer might want to make

  1. Add NumberOfProducts into the export
  2. Change the location of exported file or even the encoding.
  3. Hey. I want to see reports of 2 days instead of just today.
  4. ….

Things go on in various ways. Finally, they all come to you, developers, to reflect the changes in code. If the application is just a simple stupid console app in this example, there is not any problem at all. Everything can be changed in a matter of minutes.

But we all know that it is not true in the real life.

Embrace Changes

I would prefer to do some logical analysis before actually writing code. If I start my code first, it will affect my thinking process. Sometimes it helps.

To embrace changes, we, first, must identify the possible changed areas. Isolate and Conquer approach

  1. Identify the possible changed areas
  2. Isolate them. Mean you can conquer them. Because you know where to look at, where to change the code.

Identify

Next and hardest question is

How to identify those areas?

Answer

Think in process and interaction. Capture concepts and patterns

Back to export order example. What does it mean? How do I apply in that example?

What is the process of exporting order?

  1. Get the orders
  2. Build data
  3. Write data into CVS file
Export orders process
Export orders process

Obviously, the process will not change. Because we control how we want to implement it. In other words, it does not depend on the customer. What changes? The 3 boxes: Get orders, Build data and Write to file.

Let’s go through each of them

Get Orders

  • Where to get them?
  • How to get them?
  • Criteria

Build data

  • How many columns do we need?
  • What is the format of currency, datetime, … ? the presentation of the data

Write to file

  • What deliminator to use? “;” or “,”? Maybe directly to Excel file?
  • Where to store the file? in which encoding?
  • What if the file fails to save due to permission?

Let’s say we have identified them. Now, let’s isolate them.

Isolate

I will modify the code as below

    class Program
    {
        static void Main(string[] args)
        {
            var orders = GetOrders();
            var data = BuildData(orders);
            WriteToFile(data);
        }

        private static IList<Order> GetOrders()
        {
            return new List<Order>
            {
                new Order{Customer = "Batman", TotalCost = 100000, OrderedAt = DateTime.Today.AddDays(-3)},
                new Order{Customer = "Superman", TotalCost = 1000, OrderedAt = DateTime.Today.AddDays(-2)},
                new Order{Customer = "Spiderman", TotalCost = 100, OrderedAt = DateTime.Today.AddDays(-1)}
            };
        }

        private static IList<IList<string>> BuildData(IList<Order> orders)
        {
            // Initialize rows with header
            var rows = new List<IList<string>>
            {
                new List<string> {"Customer", "Total Cost", "Ordered At"}
            };
            // Add rows data for each order
            foreach (var order in orders)
            {
                rows.Add(new List<string>
                {
                    order.Customer,
                    order.TotalCost.ToString(CultureInfo.CurrentCulture),
                    order.OrderedAt.ToString(CultureInfo.CurrentCulture)
                });
            }
            return rows;
        }

        private static void WriteToFile(IList<IList<string>> data)
        {
            const string deliminator = ";";
            var fullPathToCsvFile = @"d:\export\orders.csv";
            var csvData = data.Select(a => string.Join(deliminator, a))
                .ToList();
            File.WriteAllLines(fullPathToCsvFile, csvData, Encoding.Unicode);
        }
    }

Each method in the Main class encapsulates a concept, area that we have mentioned. Whether we should move them into separated classes is not that important. But yes, you should move them when in production code.

With that in place, when clients want to change something, you are more confident to embrace them. Or when there are bugs. In the beginning of the post, I mentioned that clients want to add NumberOfProducts into the report. Do you know where to change? Yeah! You got it – change in the BuildData method.

The above refactoring is good to embrace changes. However, we can make it better. I leave it to you if you wish to challenge yourself.

Takeaway

Take the “Embrace changes” seriously with you. Train your mindset into the concept. As a developer, we must be aware of that simple fact. Complaining about them will not solve the problem. You will have to deal with it anyway. Better, you should be in proactive position. Changes come! Not a problem!

When implementing a new feature, write new code, I would suggest that you stop for a moment, draw the process, the interaction in your head (better if you do it on paper) before you actually write code. Once you have done that, the possible changes areas emerge. Capture them immediately.

I have a simple process for you

  1. Think about the process, the interaction in normal language. Not the language of the code.
  2. Identify the possible changes areas.
  3. Capture them immediately. You should write them down in the paper.
  4. Write code.
  5. Design patterns, best practices come last.

I hope you enjoy the reading and use them for your next piece of code.

Have a nice day!

Implement Audit Log for Legacy Code

Given a legacy project, the code has been developed for 6 years, the requirement is to log all the changes in the system. For example, if a user changes his street name, that must be logged so that the administrator can know the old and new values.

The project is a typical codebase where there is domain model, SQL Database using NHibernate framework.

The question is how do we implement that requirement without polluted the domain model, without making breaking changes in the design and functionalities?

Assuming that we have a super simple domain model with User and Address.

    public class User
    {
        public Guid Id { get; set; }
        public string Name { get; set; }
        public ResidentAddress ResidentAddress { get; set; }
        public IList<VisitedAddress> VisitedAddresses { get; set; }
    }

    public class ResidentAddress
    {
        public string HouseNumber { get; set; }
        public string StreetName { get; set; }
    }

    public class VisitedAddress
    {
        public string HouseNumber { get; set; }
        public string StreetName { get; set; }
        public string Country { get; set; }
    }

    public class SuperCoolService
    {
        public void ChangeUserResidentStreet(User user, string streetName)
        {
            user.ResidentAddress.StreetName = streetName;
        }

        public void AddVisitedAddress(User user, VisitedAddress visitedAddress)
        {
            user.VisitedAddresses.Add(visitedAddress);
        }
    }

A user has a residential address and a number of visited addresses. Whenever a user changes his street name, audit log it. Whenever a user visits a new address, audit log it. You got the idea.

Capture Changes

We, unfortunately, cannot redesign the domain model. And we cannot go into every single method and detect if values change. No, that would pollute the domain model and destroy the design. Audit Log is a system concern, not a business domain concern.

It turned out that we can accomplish that goal with the beautiful design of NHibernate. In short, NHibernate allows us to hook in its event pipeline. It allows us to add custom logic for pre and post of Insert/Update/Delete an object. Check more detail from NHibernate document website.

When designing systems, I prefer the explicit approach instead of implicit. I want to have the power of deciding what objects I want to audit, at which properties. Life is so much easier if you are in the control. The same goes for code.

    public interface IAuditable
    {
    }

    [AttributeUsage(AttributeTargets.Property)]
    public class ExplicitAuditAttribute : Attribute
    {
 
    }

    public class User : IAuditable
    {
        public Guid Id { get; set; }
        [ExplicitAudit]
        public string Name { get; set; }
        public ResidentAddress ResidentAddress { get; set; }
        public IList<VisitedAddress> VisitedAddresses { get; set; }
    }

    public class ResidentAddress : IAuditable
    {
        [ExplicitAudit]
        public string HouseNumber { get; set; }
        [ExplicitAudit]
        public string StreetName { get; set; }
    }

    public class VisitedAddress : IAuditable
    {
        [ExplicitAudit]
        public string HouseNumber { get; set; }
        [ExplicitAudit]
        public string StreetName { get; set; }
        [ExplicitAudit]
        public string Country { get; set; }
    }

All need-audit-log classes are decorated with the IAuditable interface. Auditable properties are marked with ExplicitAudit.

    public class NhAuditUpdateDomainHandler : IPostUpdateEventListener
    {
        public void OnPostUpdate(PostUpdateEvent @event)
        {
            var auditable = @event.Entity as IAuditable;
            if (auditable == null)
                return;
            var dirtyFieldsIndex = @event.Persister.FindDirty(@event.State, @event.OldState, @event.Entity, @event.Session);
            if (dirtyFieldsIndex.Length == 0)
                return;
// Custom logic to handle audit logic
        }
    }

Depending on your domain, your audit logic, the detail is yours. By implementing an IPostUpdateEventListener, we can extract all the [ExplicitAudit] properties with their old and new values. Pretty neat design.

Give It Meaning

Soon we hit a problem. We knew the street name was changed. However, we could not know to whom it belongs to. We cannot know the parent of the address. The current design only allows us to know the address of a user, not the other way around.

// Reality
Street name change from "Batman" to "Superman"

// However, we expect this
User: Thai Anh Duc
    Street name change from "Batman" to "Superman"

Think about the situation where you have many changes. The information is useless.

Context. The address is lacking Context. At the time of audit log, it needs to know who it is a part of.

The question becomes how do we give that information for the Address class without introducing a hard reference back to the User class?

Explicit Tell

The goal is to minimize the impact on the existing codebase, to prevent changes in business logic. We cannot support the audit log feature without touching the existing code. I decided to categorize the domain objects into 2 categories: ProvideContext and DependableContext

    public interface IAuditContextProvider
    {
        AuditContext ProvideContext();
    }
	
    public interface IAuditDependableContext
    {
        AuditContext ProvidedContext();
    }
  1. Context Provider: Will tell the infrastructure its own context. It acts like a root. It is the User class in our example.
  2. Dependable Context: Will tell the infrastructure the context it is provided. This allows the infrastructure links the context. It is the Address class in our example.

I must write some code to build the infrastructure to link them. It is infrastructure code, not interfere with the domain code.

There are 2 scenarios we have to deal with: Component and Reference relationship.

Component Relationship

In NHibernate with SQL, a component relationship means 2 objects are saved in the same table. Here is the detail document from NHibernate site.

User_Address_Component
User and home address are merged into one table

Because address cannot stand alone. It cannot be added directly to NHibernate’s session. This gives me a chance to intercept its parent (the user object) when it is attached to NHibernate’s session. I just need to find a way to tell the NHibernate: Hey, whenever you save a user object (via its session), tell the user to populate the audit context to its components.

Thank the beautiful design of NHibernate, this code works beautifully

    public class ProvideAuditContextSaveOrUpdateEventListener : ISaveOrUpdateEventListener
    {
        public void OnSaveOrUpdate(SaveOrUpdateEvent @event)
        {
            var mustProvideContext = @event.Entity as IMustProvideAuditContext;
            if (mustProvideContext != null)
                mustProvideContext.ProvideAuditContext(NHibernateUtil.IsInitialized);
        }
    }

What the code says is that “Hey, if you are a ‘must provide audit context (IMustProvideAuditContext interface)‘, then do it”. When the entity comes to the later phases in the pipeline, it has the audit context provided.

Reference Relationship

However, if the Address object is a reference object (in which it is saved in a separated table), it will not work. Because the referenced objects are inserted first. Take an example, we want to allow a user having many addresses. In this situation, User has its own table. And Address has its own table. There is a UserId column in the Address table.

User has addresses
User has addresses

Take an example, we want to allow a user having many addresses. In this situation, User has its own table. And Address has its own table. When a new address is added to a user, that address is saved first. Then the user is updated.

Because Address knows its user, we can take an advantage by overriding the ProvidedContext method

    public override AuditContext ProvidedContext()
    {
        var context = base.ProvidedContext();
        return context.ContextId == Guid.Empty ? new AuditContext{ContextId = UserId} : context;
    }

If it has a context, then use it. Otherwise, create a new one with UserId

Takeaway

When starting a new project, audit log might not be mentioned, might not be seen as a feature. We, as a developer, should ask customers, managers if we should concern about the Audit Log. No matter the answer, you should be aware of its coming. You should consider Audit Log when designing the code.

Audit Log is an infrastructure concern. Make sure you treat it as is. A very bad approach is to go in every single method call in the domain and register an audit log (or any kind of audit). There are better ways to deal with infrastructure concern from Domain code, depending on the framework you used. In my project, I accomplish that goal by combining

  1. Attribute: By decorated a property with [ExplicitAudit] attribute, I control what properties I want to audit.
  2. NHibernate Events: A well-designed framework will allow you to hook into its pipeline and add your custom logic. I take advantages of NHibernate well-designed events. If you use EF, I am sure you will find many places to hook your custom logic in.

Have you seen the power of OCP (Open-Closed Principle) in NHibernate design? Such a powerful principle.

The real implementation is far more complex than in this post. However, the principles stand. It was really a challenge.

So far so good. It is not a perfect implementation. But I am happy with the result.

Deliver This Feature, Agile Said. Hmm, What About?

Agile is a good methodology to get things done. I buy the philosophies behind it. I think it can be applied in many areas, not just at work, not just at building software. To me, Agile is a tool, a meant to help me get the job done. In software development, I have heard people talking about these beautiful things.

  1. Agile team is self-organized
  2. Deliver working software (features) every sprint.
  3. No upfront design.
  4. Hmm, do we need an Architect? Do we need a Team Leader?
  5. Embrace changes
  6. ….

Many fantasy things. When I first heard about them, I thought

Wow. That’s so cool. Everything looks perfect. Hmm, if everything is perfect, then there is a hidden problem.

Unfortunately, I felt something was missing. Later on (at least up til now), I have found many. Basically, all those things are correct if, and only if, you have a perfect Agile Team. In reality, we all know that human is the root of everything. And having a good team is, well, just a dream. We have to face that truth. Do not try to assume we have a good team at hand. It just does not work that way.

Let’s take a small area from that fantasy: Add a new feature.

The Story

Good morning SWAT Agile Team, in the sprint we will deliver a new cool feature: Delete (soft delete) user. The feature will allow the administrator to delete a user. Users are marked as deleted but not removed from the data storage.

With these facts

  • The system has 10 of features built in.
  • There are features associated with User: Create, Edit, Manage Order, …

The wishes

  • The new feature will not break the existing ones.
  • Reuse the code as much as possible.

The untold facts (which will be discovered when writing user stories for the feature)

  • Some existing features will be impacted. Cannot delete users having orders. Cannot edit a deleted user.
  • Code Quality: Cyclomatic Complexity (CC) must be under control.

If you have been in such a situation, please share how you will handle it.

The Problem

Developers, Testers when joining a so-called agile team, how much have you known about Agile? When a project starts, developers are sold a plan of getting small deliveries, increasements. No upfront design. No architecture. Hey! That is the waterfall, folks, an Agile-men said.

There are many requirement meetings, discussions, … Everything goes well as planned. The team has a clear set of user stories. Estimation is done. Agile is on set.

Good! Ladies and gentlemen, where do we write the first line of code? Serious. Where do we write? Not mention how. To make thing super clear, where/how do you write the first line of production code? Just to opt out the Unit Test code.

A harder question. How do we code in such a way that in the middle of the sprint (or next sprints) the requirements change? How do we embrace the changes?

As the system grows, the harder to add a new feature.

The Solution

Hell No. I do not have a solution. However, I do have some proposals.

Infrastructure and Architecture

No matter what fields we are in, to embrace changes, to stand still, we need support from Infrastructure and Architecture. Without a good foundation, we just build a house on the sand. A single wind can blow it away.

We need an architect. We need a good, stable foundation. If there has not, we must build one.

We need an architect to define system boundaries, to separate the logics into proper concepts, to decide the architectural style, to decide design patterns. A good architect will limit the impact when implementing a new feature or modifying an existing one.

For legacy systems, a good architect can help decouple the dependencies, create boundaries. It is much harder than in the green field projects. But it can be done and can improve over the time. Otherwise, every single change is a risk.

Agile Mindset

People are the core of everything. Members decide the success or failure of the project, of the team. What is their mindset? We bring Agile in to increase the flexibility, to embrace the changes, … None of them will work if members think

  1. I must have a clear requirement to start with.
  2. My job is just coding or testing
  3. Who cares about the design. I just need to code this function. I change only a small part of the code.
  4. Hey, I just add a new method for me. Isolated. No breaking change.

Still the old mindset. I am not saying that mindset is wrong. But it does not fit the new environment either.

I would suggest the first thing we should do is to train our member; to get them adapt with the new mindset; to install the agile mindset into their subconscious mind. It is not easy, not at all.

A few things we can train developers

Architectural Thinking

There is no clear definition of Architect. However, everyone can think architecturally. Every developer, paid for a job, must be able to write code. They should not stop there. Instead, ask these questions

  1. Can I read my code after 3, 6 months? How about a year?
  2. How many ways can break the code?
  3. How easy to change? What if clients want to change XXX?
  4. Is it easy to understand?

What kinds of architectures that support well to embrace changes?

It is a long road. It is worth a try. Oh, wait a minute! Which means we need an architect. Yes, Sir! We need an architect or one having architecture skill in the team.

There is an argument saying that the architect will setup the system, build the framework, make some training sessions, and finally handle over to the team. Good luck! Soon enough, the framework will break. Things will tear apart. Why? Because of the lack of quality of maintainers, the development team.

Coaching

We cannot change someone else mindset. It does not work that way. However, we can show them the right mindset. We can consistently tell them the right mindset. Coaching approach might work. That is you do not train or tell someone what to do. No! They decide what they want to do and how they will do it. We, the team leader (I hope you have one), will

  1. Give some suggestions, show the direction
  2. Encourage
  3. Motivate

Will It Work?

I do not know. But if your current team does not work as you expected, maybe you should try it. There is no silver bullet. Try and Improve!

In discussions, every leader seems to know that people are the most important asset. However, when it comes to daily work, the reality might not the same. How much do we care the most important asset? What have we done to improve that asset?

Agile or waterfall, no matter what methodologies we use, we are a team works on a mission. We want to get that mission done and we want to be proud of what we have done.

Should we need an Architect in the team? Yes. Definitely. Then help everyone think as an architect.

Should we train our people? Yes. Definitely.

To win the war, you need good soldiers!

Happy Sunday! Thank for your reading.

Code Duplication is a Myth

I know that code duplication is not good. We try to avoid code duplication. In a perfect world, you do not have code duplication. What annoys me is that we seem to forget to ask some critical questions to ourselves.

  1. Is it really that bad in all circumstances?
  2. What might be worst if we try to avoid code duplication?

Let’s take a typical web application where you have features to add and edit a user.

With 2 separated models

    public class CreateUserModel
    {
        public string Name { get; set; }
        public string Address { get; set; }
        public DateTime? DateOfBirth { get; set; }
    }

    public class EditUserModel
    {
        public int Id { get; set; }
        public string Name { get; set; }
        public string Address { get; set; }
        public DateTime? DateOfBirth { get; set; }
    }

Or one single User Model?

    public class UserModel
    {
        public int Id { get; set; }
        public string Name { get; set; }
        public string Address { get; set; }
        public DateTime? DateOfBirth { get; set; }
    }

In single user model, if the Id is 0, assume that is creating a new user.

In the real world application, the model is complex. I make this example to demonstrate the point: There are many duplications in 2 models. The question is whether it is good or bad?

Well, it depends … on the context.

Why do we see it duplicated? Mostly because the code looks the same. I, however, want to look at another angle, from the purpose and context angle.

In term of business view, create and edit a user are 2 different features. They operate on the same data object (User). But they are not the same. Let’s assume that we have a web application. There are 2 screens, one for Create and one for Edit.

        public IActionResult CreateUser(CreateUserModel model)
        {
            return View();
        }

        public IActionResult EditUser(EditUserModel model)
        {
            return View();
        }

        // Reuse the same model for 2 different features: Create or Edit
        public IActionResult SaveUser(UserModel model)
        {
            return View();
        }

In the SaveUser action, if the User.Id is 0, use the Create logic, otherwise, use the Edit logic. So we can put aside the logic.

Initially, both Create and Edit screens look the same. They operate on the same piece of data. The single model sounds a right choice. However, I will choose the 2 models. I do not see them duplicated. I do not see benefits of unifying them.

Here are why

  1. Each is an independent feature.
  2. They have different input and output. Sooner or later you will end up some information on a feature not make sense on another.
  3. By separating them, we can have 2 teams (groups) work on 2 features without stepping on each other toes.

Ok then, when will it make sense?

I will consider a duplication when

  1. A piece of code that carries logic. Not just data or POCO.
  2. Independent of context. The refactor code (as a result of killing duplication) might depend on the input parameters. However, keep an eye on the number of parameters and make your instinct decisions base on your experiences.
  3. Easy to understand. There should be a very limit of condition/switching in the implementation.

From my experience, killing duplication is not an easy task. It requires a lot of skills

  1. Skill to ask the right question about the value of the code, the logic of the code, … One should ask themselves serious questions.
  2. Skill to read the code and get a brief of what it does.
  3. Skill to capture just enough to refactor out.
  4. Skill to refactor.

Did I tell you that Naming is one of the hardest tasks? Given that you have a piece of duplicated code, what would you name them?

I fell in love with the slice (with Feature approach) architecture when I first saw it. I saw its value when traditional architecture caused my headache when the code was out of control. Especially, when I have to develop a new feature. The biggest fear when adding new features is the chance of breaking existing features, aka causing regression bugs. With the feature approach, we can limit the risk.

Every approach has its pros and cons. We, developers, have to weigh them and pick the one that fits most.

There are some valuable links from experts.

Jimmy Bogard SOLID Architecture.

Steve Smith ASP.NET Core Feature Architecture.

Application structure: Concepts and Features.

Isolated Feature from Ayende.

Or checkout Jimmy repo here. A wonderful resource to learn how to build a web application with the Slice + MediatR + Feature.