Lotushints

It was Saturday. I distinctly remember. I was still knackered when I woke up. It shouldn’t have surprised me, really. I had been pulling overtime entire week. Again. As I usually do, I checked my side project apps before the kids were up in the morning. And what do you know?! One was unreachable. Turns out I had forgotten to enable “automatically extend Let’s Encrypt certificate” option for one of my apps and certificate silently expired.
That is the thing about certificates. They expire silently without any warning. Something just breaks. Preferably crucial and in production. And then you are on the frenzy to find out what went wrong.

I normally use calendars for this. But calendars have a habit of notifying you, when you are out and about with your family. And when you mark that notification as read? Gone. For good.
I needed and wanted something better. Something that would send an email or a slack message. Something that sticks. Not just a notification to my phone.
So, I built CertWatch. And unlike what you expect, this isn’t a post about TLS or certification expiry. Nope. This is a post about how “Pffft, I can do this in a month” turned into seven months of early morning hacking, while having full-time job and handling family life to get to “closed beta”.

The beginning

Idea for the app is simple enough, to make me believe that it could be done easily and quickly. A user would just enter a URL and application would obtain public key from certificate and store it. Then, when it expired or changed, a user would get notified about it via e-mail. It was March. I was sure, I can get it done by end of June. Easy, peasy. As long as I didn’t procrastinate.

There were just a few constraints. Full time job, family and about an hour of spare time to actually make something. The app had to be in .NET 10 using Microsoft MVC. And since I am hosting everything on a shared hosting, I had to use an embedded database.

I decided to use AI, but not for coding. Instead, I would use it as a personal fan club: tracking my progress, kicking my butt when I procrastinate and telling me I was the best in the world in between (obviously). Also, I use AI to learn marketing, because I really really suck at it. It worked. Most of the time.

The first month went on basics, like idea validation, database model and setting up basic application. The easiest was database model. I already knew what embedded database to pick (SQLite d’oh). Idea validation took time and a lot of reading. I also managed to find a few users interested in my non-existent app. Also, I created a very basic web site and published it.

Unsurprisingly, setting up the basic app took most of the time. Version 1.0 was a simple, single-user app that notified me when certificate expired. I had to enter certificate manually and directly into the database. But at least I had a script I could use to create a version.

I also did light Twitter marketing that yield about a hundred views in total and 0 traction. LinkedIn was a tad better and got a few people on my wait list.

The work continues

Encouraged by LinkedIn success, I spent April and May making sure CertWatch supports multiple users. I also created option to enter and view APIs being monitored. Still no login, though. But at least the app displayed errors nicely.
In June, work and family life kicked in and time got sparse. In first half of the month, I only managed to do some maintenance work on the part that handled expiration notifications and cleaned up the code. Thankfully, things changed and by the end of the month, I finished Login page. It took database model changes and a lot of elbow grease to finish. The app could login users now. I had no way to add them except by inserting them into the database by hand.
And this was Version 1.01.

July was spent on vacation, family and work. and the app lay forgotten until month of August. It was time to focus on user security features. Since I was entering users by hand, the app desperately needed a password change option. And I also had to redo the entire navigation, since navigation up till now had only been “Logout”. And that was basically it for August and Version 1.02.

The only thing left, was to give wait-listed users access and contact them. I assembled a nice email, sent it over and… none of them even tried to log in.
I spent zero time on marketing, which in hindsight was not the best idea. I did manage to convince a good friend to try it out for his own projects. So, my app now had two users, including me.

First bugs and improvements

Encouraged by getting one other soul on board, I started September strong. First, I needed to fix bugs and two of them were nasty. One was related to displaying error when trying to log in with invalid password. In production environment, the login error did not show. Users got 404 error instead. It ended up being a stupid redirect mistake, but it took time to find.
The other bug was related to storing certificate expiration dates in local time instead of UTC. A rookie mistake that took way too much time to fix. Version 1.10 was ready by mid September.

After that, my focus shifted to improve UX. Until now, users had to enter URLs with “https://” in front of them. This was unnecessary, as you cannot check a certificate over http. So, I removed that.
I also added sorting for entered URLs and a page showing certificate details including public key download. I am proud of that.

For once I at least pretended to do some marketing and for Version 1.11 that was released by the end of September, I also added a screenshot of the app to app’s web page and did minor tweaks to subtitle and call to action button. I pinged LinkedIn wait-listed people again and was met by a wall of silence.

User registration

The app still had no self-registration. And this is exactly what I have been working on in October. But, since I don’t want any odd spammer to gain access, I decided registration should require a promotion code. Which filled my to-do list. It is surprising how much time and energy a simple thing like promotion code generator and consumption takes.
But, come October 16th, Version 1.12 was released and users can finally register for themselves. Provided they somehow obtain a promotion code.

If you want to try it…

CertWatch is a quiet little tool for a quiet little problem: certificates that expire whether or not you remember to check.

If you run services, a side project, a company, anything with an HTTPS endpoint, I have a small number of closed-beta codes. Each one lets you set up an account, add your interfaces, and get notified before anything expires. Go to link below and click “Request access” button. No matter what the page says, you don’t have to pay to be a closed-beta user.

Get a closed-beta code

In the previous article “.NET Certificate Pinning: A Practical Guide“, we implemented simple certificate pinning in a command line .NET application. You went and implemented it in your app. You tested it and it worked perfectly.

Fast forward a few months and your users cannot connect. You haven’t changed the code for ages. So you check your API and it works fine, but your app not so much.

The problem: certificate rotation. Something completely outside your control, that managed to break your application. Most likely in the most awkward way and at the least desirable or expected time.

What is certificate rotation?

Certificate rotation is a process of replacing existing certificate with new ones. Certificates require rotation due to following reasons:

• certificate is bound to expire (some more often than others and yes, I am looking at you Let’s Encrypt),
• certificate has been revoked,
• there is a need for improved algorithm or
• domain changes.

Expiration you can plan for. Other rotations are silent, unpredictable. And by the time you notice, users have already noticed it. By then, it is too late.

Why my app breaks?

Certificate pinning is based on certificate public key, hash, trust chain or other metadata. When certificate changes, some data (if not all) changes as well, causing the failure. Certificate hash and public key change with each new certificate. Trust chain changes less frequently, but pinning only the chain is insecure and weak. And that defeats the purpose of certificate pinning.

What can I do to avoid downtime?

As mentioned, the easiest thing to do is to set up expiration reminders. That can be easily done via a calendar.

Some certificate authorities (CA) allow renewal with the same key pair. If you can control the server and configure this, the certificate’s public key will remain unchanged. But by default, many renewals generate a new key. Still, pinning Subject Public Key Info (or SPKI) can be a decent solution. SPKI is the part of the certificate that contains the actual public key and its algorithm. Pinning this survives renewal, if the key pair stays the same.

Also, implement backup pins. Keep at least two: the current certificate and the backup. You can use your CA root or intermediate certificate for this. When rotation happens, your application still trusts the backup while you update the primary certificate. You will need to refresh the list over time, but users will not experience downtime.

This is exactly why I am building CertWatch. The app tracks your certificates and alerts you 90/60/30/14/7/1 days before expiry or immediately on unexpected rotation event, notifying you before your users do. The alert includes the new public key so you can update your pinned app before they break.

Conclusion

Certificate pinning makes your app more secure, but without handling rotation, it makes your app brittle and prone to unexpected outages. Pin SPKI, keep backups and set reminders. Do that and you won’t be waking up to a bunch of angry emails and calls in distress.

In the previous article “Certificate Pinning: Why Your App Needs It“, we discussed why certificate pinning is essential for securing client-server communication. This article will focus on pure technical side of certificate pinning implementation.

We will create a simple command line application that fetches source code of my SaaS product web site CertWatch. Why command line? Because, it is good enough example that will work everywhere else and because it is isolated enough that it is easy to follow even for a casual tech follower.

The site uses Let’s Encrypt certificate, that changes quite often, so beware that certificate public key may not be valid at the time you read this article.

First, we need to obtain public key of server certificate. We can do this by either viewing certificate in the browser or use a tool like openssl.

private static string g_ssPublicKey =
		"3082010A0282010100C4FFFFE6F3945AD2BF27D5DD674166130D5D2021CEFF0" +
		"4B06AD48F5F56A6245C590433121C8F08B7A565FBF38F1102917CB7434AFE91" +
		"18E2CB904BA723D57182B680B872CF05578B234F65DB1A39CD77DEBD07D0939" +
		"A0C440A9AE9245D0CAB59480DC3864D744BA6404B0D6DA9BAEE0E85CE0816D9" +
		"D7F43468D2E073CBA2EA10114323B0053F8AE29F86AD846B71FE4D7924494FB" +
		"0D80E3C78875085163B53121EBEBCF1356A4386DFF9E2CB93D0BD9CA3A39D4A" +
		"AC7BB34F2FF4AC70D59DBCD92254D48DE0BC3CCB4A8B4822D64CCE46F1E539B" +
		"116A00420825AD2AFF128F7A761D79186FB747761E47187BD527B1398F603DC" +
		"F7DCABD3535C28B7FB2C3068230203010001";

In the program Main function, we will use HttpClient to connect to the web site. In order to set certificate handling and verification, we must first declare HttpClientHandler. The handler must do at least three things:

• enable Certificate Revocation List check to check that certificate has not been revoked
• set client certificate options to Manual, as we are going to manually validate certificate
• implement server certificate validation callback method to perform validation.

Before we implement the callback method, let’s set a boiler plate for it and set first two items on the list. Validation method will return false and thus fail on every request.

using var handler = new HttpClientHandler(); 
		handler.CheckCertificateRevocationList = true;
		handler.ClientCertificateOptions = ClientCertificateOption.Manual;
		handler.ServerCertificateCustomValidationCallback = (message, certificate, chain, sslPolicyErrors) => { return false; };

Great. With that done, we can implement the body of the callback function. First, we need to check if the server web site certificate actually exists. If it doesn’t we return false and print an error. Here, you would do some logging instead, but this is a demo app.

if (null == certificate)
{
	Console.WriteLine("ERROR: No server certificate found");
	return false;
}

Next, we verify there are no SSL policy errors in certificate chain. If they are, we return false and print all policy errors.

if (sslPolicyErrors != SslPolicyErrors.None)
{
	Console.WriteLine($"SSL Policy Errors: {sslPolicyErrors}");
	return false;
}

Then, we obtain, the public key of server certificate and check if it matches the one, we declared in our app. If there is no match, we again print an error and return false. If public keys match, then the certificate is correct and we successfully end validation.

string sPublicKeyToVerify = certificate.GetPublicKeyString();
if (g_ssPublicKey != sPublicKeyToVerify)
{
	Console.WriteLine("ERROR: Certificate public key mismatch");
	return false;
}

Console.WriteLine("Certificate public key matches");
return true;

Now, all we have to do, is call our website and print it’s source code. The validation procedure will run immediately after SSL handshake is established.

using var httpClient = new HttpClient(handler);
string sResponse = await httpClient.GetStringAsync("https://certwatch.dev");
Console.WriteLine(sResponse);

In less than 50 lines of code, we have implemented a working, robust certificate pinning that ensures our app only communicates with our genuine API server. This simple technique significantly raises the bar against possible man-in-the-middle attacks.

Complete working example is embedded below. If you cannot see it, it is also published as gist on Github.

In the next article, we will check common pitfalls and dangers of certificate pinning in production environments.

Today, we, developers have it easier than anytime before. The AI bubble was supposed to replace us or at the very least, lower our hourly rate. Instead we have gained even more flexibility and speed. Always improving AI agents lured a lot of people to coding. The number of apps released each day went exponential. But speed shouldn’t come at a cost of security. Not when so many people can get affected.

Most apps follow a typical client-server setup and in that setup, you just cannot assume that every response your app receives actually comes from your back-end server. Attackers can spoof your API’s URL, intercept traffic or revers engineer your client to launch man-in-the-middle attacks. Never assume that because you are nice, the internet is too.

One of effective approaches for verifying who your app is talking to, is called certificate pinning. It might sound fancy, but the idea is simple: ensure your app only accepts connections from servers using an expected certificate or public key.

When your client app connects to your API over a secure connection (HTTPS), it checks that the server’s certificate is valid and issued by a trusted certificate authority (CA).

By implementing certificate pinning, you add another check. Your client app verifies that the certificate or its public key matches the one that your client app expects. If it does not match, the connection is rejected. Even, if the certificate is otherwise valid.

Couple conditions have to be met, for you to be able to implement certificate pinning:

• your server must use secure connection (HTTPS) with a valid certificate from a trusted CA and
• you need to embed the certificate’s public key or its fingerprint in your client app.

For public apps, certificates from providers like Let’s Encrypt are totally fine. For internal web apps, a certificate can be issued by any CA that your users computers trust. It still needs to be valid though. Do note, that mobile apps don’t automatically trust private CAs and it can be a hassle to install one on mobile devices.

So let’s say you got everything set up. The only thing left to do is to implement certificate pinning. First, you need to extract your server certificate’s public key (or its fingerprint). You can do that using your browser (by navigating to your API address) or tools like OpenSSL. Then, you embed the value in your mobile, web or desktop client and add logic to compare the server’s certificate against the embedded value during the SSL/TLS handshake. Most platforms have libraries that simplify this.

You have to take into consideration that a certificate will expire. Thus, you will need to release new version of your app to support newly issued certificate. Also, make sure you test well in testing or staging environments as you can easily lock users out of your application, if pinning is not done correctly.

Certificate pinning is a straightforward but powerful way to boost your app’s security. It ensures that only your API can talk to your app.

If you are curious to try it yourself, there are great tutorials online. In next post, I will present a simple certificate pinning implementation in .NET command line app. Stay tuned.

I develop mostly in safety of Visual Studio or Visual Studio Code. And in .NET. Only recently, I rediscovered passion for C++. The speed, the elegance, numerous hours thrown into the thin air trying to find a solution to a simple problem, that somehow is just notoriously difficult in C++… You get the gist of it.

Visual Studio is a wonderful IDE. Mind you, due to it’s heavy weight and feature-rich nature, my development process was always limited to my desktop PC. And the same lately goes for Visual Studio Code. Even though, majority of my work and side-projects is done on desktop PCs, I do want to sometimes work on stuff, while I am nowhere near my home and my PC. For those situations I own an entry level Surface Go 2 with some really really shoddy hardware. It kind of holds it’s own for browsing and remoting to a work PC, if there is an emergency, but installing Visual Studio or Visual Studio Code on it is out of the question as there is not nearly enough diskspace, CPU power or memory for that.

I did think to crate a VM on my home lab server to run Windows and mentioned IDE, but I really don’t want to waste so much resources just to use it occasionally. Plus, I don’t like to expose my home network to outside incoming traffic.

But, I don’t need a full fledged IDE to develop stuff. I mean, it is a nice feature to have, but realistically, all you need is something with basic highlighting and formatting and a decent compiler. I cannot imagine this tablet to compile anything dotnet related in reasonable amount of time, however, it should be totally fine compiling C++ code. I mean, if C++ and C code compile reasonably fast on my Pi Zero 2W…

Hence, I went with Notepad++ as my editor, MinGW as compiler and cMake as build tool. And I lost a day just trying to build a Hello World project. I never used MinGW or cMake, but I heard good things about them from nice folk of the internets.

MinGW

Installing MinGW is a piece of cake. I just typed MinGW in my favorite search engine, clicked first link and downloaded it from SourceForge. It was only after several hours of trying to make cMake cooperate with MinGW, that I noticed, my version of MinGW is totally wrong. You see, MinGW is stuck at version 6.x and is not maintained anymore. In order to get latest and greatest features, what you really need is MinGW-64. So I removed old MinGW and went to the page, looking for install package. Except, binaries on Github are not latest and greatest and, according to VirusTotal page, contain a trojan. Hence, I went the way, VS Code page recommends. Installing MySYS2 first and then use pacman to install MinGW-64.

cMake

For those who do not know what cMake is, it is basically a tool that helps you generate make files and build your project using your desired compiler. It supports wide variety of compilers including VC++ and MinGW.

Installing cMake was way easier. I downloaded and installed a package from projects official site. Simple enough. The problem with cMake though, is that it has some steep learning curve and subpar documentation. Specially, if you are a beginner, you will lose numerous hours, for simple tasks that could be explained in one code snippet.

How cMake works in a nutshell is, you need a CMakeLists.txt file in your source directory. That file tells cMake, what should compile, using what compiler and linker and which libraries should include and whatnot. Of course, to make it a tad more interesting, you must use cMake scripting syntax. Have I mentioned, their documentation is really subpar?

Putting it all together

For demo purposes, I created a simple Hello World program in C++. I named the file main.cpp, as that is how you named main CPP file when I started to learn the language. You can name it foo.cpp, if you like.

#include <stdio.h>

int main()
{
    fprintf(stdout, "Hello World!\n");
    return 0;
}

Next to that file I created a file named CMakeLists.txt with following contents. The name cannot be foo.txt in this case, as far as I know.

# Minimum cMake version for build process
cmake_minimum_required(VERSION 3.28)

# I set PROJECT_NAME variable to example1
set(PROJECT_NAME example1)

# Set C++ standard 
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)

# Define project and tell cMake it is a C/C++ project
project(${PROJECT_NAME} CXX)

# Set different output directory
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY "../bin")

# Get all source files recursively and store it into SOURCES variable
file(GLOB_RECURSE SOURCES RELATIVE ${CMAKE_SOURCE_DIR} "*.cpp")

# Add the executable with all found sources in SOURCES
add_executable(${PROJECT_NAME} ${SOURCES})

As far as I know, that is the most minimalistic cMake script you need to compile a simple program.

To create a Makefile, I created a new directory named “build”, because I am weird that way and believe that build scripts have nothing to do with my source code. In command line, I went to that “build” directory and ran:

cmake .. -G "MinGW Makefiles"

Command tells cMake to take parent directory (“..”), search for CMakeLists.txt file and use “MinGW Makefiles” generator to generate Makefile. And it did just that. And some other files it needs to compile the project.

To compile my very interesting project, I needed to then run:

cmake --build .

And voila! About 30s later (my tablet is not THAT great) binary named “bin\example1.exe” has been built and outputs “Hello World!” as expected.

This article is in fact a second one in a Rapberry Pi drone series. Previous article can be read here.

In Part 1 I will cover basic overview of drone architecture and elaborate on components used. As the image below displays, this project will need:

• main controller,
• 2 batteries,
• 4 brush-less motors,
• 4 ESC controllers,
• power distribution board,
• accelerometer and
• barometer.

Main controller

The role of main controller will be taken by Raspberry Zero 2W. Why? I had one lying about collecting dust. Could I just as well used an Arduino? Yes.

Main controller will have to implement event loop, telemetry logging and navigation logic. It will also be responsible to talk to all components used. I am going to make it support software-in-the-loop simulations, which should make debugging of navigation logic more discrete.

Accelerometer and barometer are I2C components and are going to be daisy chained on same pins. ESCs will be connected to 4 separate PWM (pulse width modulation) gpio pins of Raspberry pi. All code will be written in C.

Controller will be powered by its own battery. This is going to be a RealPower PB-Lipstick powerbank with 2500mAh and 5V/1A output, which should suffice for about 1 hour of operation.

Accelerometer

For this implementation, I will be using Adafruit Triple-axis accelerometer MMA8451. It will be used to define yaw, roll and pitch to determine how the vehicle is poistioned in 3-d space. Also, the component can measure acceleration up to 8G, which is plenty.

Barometer

Another Adafruit component MPL115A2 will be a barometric pressure and temperature sensor. It is not the best of the bunch, with precision of 1.3kPa, but it should suffice for this implementation. The fact that is outputting temperature as well is only an added on bonus.

Electronic speed controller (ESC)

To drive powerful motors as the drone needs, a component called electronic speed controller is a must. ESC can be controlled using pulse width modulation, meaning that length of pulse defines output power of ESC, hence running motor at higher or lower speeds.

Another reason to use Raspberry Pi as main controller, as it supports 4 PWM devices via gpio.

For power, ESCs will use LiPo battery 2S / 3S.

Brush-less motor

To fly a drone, one needs at least 4 powerful motors. Classic brush motors are just not cutting it, due to low power output. Hence, brush-less motors will be used. However, power, has its down side. Each motor will use maximum of about 45W of electricity (7.4V and 6A of current).

To calculate how much power your motor needs for lift-off, I suggest you read this excellent article about this topic. For my application, 45W of power per motor should be plenty.

Power distribution board

As each motor needs its own ESC and there will only be one battery to power it all, a power distribution board is going to be used. I could have done it myself. I opted to buy one instead, to save on time and space.

Battery 2

This is the main battery for main motors. It is a LiPo battery 2S / 3S. I am leaning towards 3S due to capacity. However, higher capacity means more weight. A battery will need to be able to output 7.4V with 24A of current consistently.

In next part, Part 2, we will cover basics of how to drive I2C devices with RaspberryPi.

Lately, I just could not find motivation to work on my side-project, hence I spent my time procrastinating on a well known video sharing social website. Purely for motivation purposes that is. Yeah. That is it. Purely for motivation purposes.

During my procrastination… ummm… motivation period, I have encountered a vlog of a guy doing model rockets with active navigation system. You might have heard of his company. It is called BPS space.

Watching his videos made me think back of the days, when soldering and electronic circuits design was something I did regularly. The more I thought about it, the more I was convinced that an electronic project was something I needed to get my juices flowing again. Model rocketry is out of the question for many reasons, but I thought of another project that might be interesting. Building my own drone.

But why would I want to build my own drone? Well, one, it looks like a fun project and two, I always wanted one. Sure, I could buy one, but where is the fun in that? Plus, kids have grown to a point, where such a project might interest them as well.

I am starting small. I want first incarnation (conveniently named Lev One) of my drone (a quad-copter) to levitate in a closed space (like, you know, a room) at relative altitude of 1 meter. In the future, I will add other things, like ability to maneuver it manually, a camera and of course a “follow me” feature. But for now, getting it off the ground straight to a height of 1 meter sounds plenty. Better yet. It sounds like a manageable step. On paper.

To get a drone to levitate at 1 meter height, the drone needs to understand how high it is and what is its position in 3D space. It needs to have the ability to maintain it’s position even though a closed space likely has little if any wind effect. It also needs to have a powerful battery to drive it’s four engines and run a controller for navigation.

In order to achieve this milestone I will need:

1. a controller to handle navigation
2. some sort of barometer that will help determine altitude
3. a gyroscope to determine pitch, roll, yaw and acceleration of a drone
4. four brush-less motors to run propellers
5. four ESC controllers to control speed of motor
6. a voltage divider to deliver expected voltage and current to all hardware
7. a battery to power it all
8. some sort of housing and
9. a lot of luck and patience.

I have already decided on a controller, which, as one might have assumed from the title, will be a Raspberry Pi. More accurately a Raspberry Pi Zero 2W I had lying about for some time now, gathering dust. I have a vague idea about the feature set it will need to sport. Details will be covered in one of future articles.

These articles will be published sporadically and will cover topics as I progress on my path to my milestone – Lev One and onwards. Part 1 will cover my overview of the project and will describe all components used in this project as well as decisions behind choosing these components.

Recently, I have been dealing with too many issues regarding .NET Json serialization on one side and Javascript (de)serialization on the other, to keep confidence in this technology. From DateTime serialization issues, to the mere fact that serializing DateTime.MinValue causes a fatal crash, because UTC time is less than minimal allowed DateTime value. But the last one was a drop too many.

Before I go any further, I would like to state that said definitely applies to .NET 4.8.x. using latest Newtonsoft.Json package. I am yet to test it on .NET 5, but I have an odd feeling, the issue remains.

The problem

I lost a lot of time trying to circumvent this issue. We are signing our responses on the server and sending hash along to the client who then verifies, if the hash is valid.

Naturally, in our production code we are doing much more complex stuff than shown in this article, but we’ll keep it simple for easier understanding.

Imagine your .NET API produces simple JSON response: {"Value":385.0}. SHA-2 hash value of this is: DE2A5770B9E63DCC04B1A417E8E6DEE4E83619CA87D6A22A49CEEAC9925C6643.

This data now gets sent to JS. In order to check the signature, client code must convert JSON object back to string and calculate hash. You can do that by calling JSON.stringify() on object and use crypto.js to calculate SHA-2 hash. All fine and well, except the hash on JS client is: 99F411EF3B0CB566199EFA6835C33DE0727690325B155B4FC5C5FA8A340AA714.

Not quite what we expected. But in order to know why this happens, you need to understand…

Decimal serialization in .NET

Decimal serialization in .NET is a funny ordeal. If you read the documentation (and you really really should), you know that serialization keeps the amount of decimal places your program assigned to a variable. The reason is: “we don’t want to change your data”. That I can get my head around.

However, what is more difficult to explain is that this statement does not apply at all times. For instance, imagine you have an object with a decimal value of 385.00000. JSON representation of such object will be something along the lines of: {"Value":385.00000}. Expected and nothing special.

However, if you set a decimal value of 385M, the JSON representation is now: {"Value":385.0} and not {"Value":385} what one would expect. So much for “we don’t want to change your data” mantra.

And, if you think I did anything special about it, I present you the code that generates said result:

public class SampleObject
{
    public decimal Value { get; set; }
}

var obj = new SampleObject() { Value = 385M };
System.Console.WriteLine(JsonConvert.SerializeObject(obj));

But this only starts the problem. To know the whole store, you need to know…

JSON serialization in JavaScript

JSON serialization in JavaScript is natively supported by Json.stringify(object) method. But, using this method on above object {"Value":385.0} returns a string representation of {"Value":385}, which is not exactly expected behaviour.

The exact same conversion happens no matter how many decimal zeros your value has. So if your object is {"Value":385.00000}, JSON.stringify of that object will return {"Value":385}.

If you check Google on this, you will get the answer of typical arrogant programmer (which is the problem with a LOT of JavaScript features): “385 is exactly the same value as 385.0”. True. Except, when you try to check a digital signature of passed data. Then, 385 and 385.0 are as different as night and day.

And now, we get to the trickiest part of them all…

How do I circumvent this?

First, JavaScript gives you no weapons to get rid of this. Except for some iffy, string replacement techniques.

Hence, your only option is to format the response “properly” in .NET code. Except, this cannot be done straight-forward, as as we have seen, serialization of 385M serializes into 385.0.

Lucky for us, Newtonsoft.Json library offers the ability to write custom JsonConverters. Finally some good news. We “only” need to write our own converter. But how do we convince our converter to use only as many decimal numbers as needed? A lot of “googling” later, there seems to be about two passable solutions. On contains formatting the number as “G29” format. The other contains division of value with 1.00000000000000000M. Both produce similar results.

I started typing and several minutes later, I ended up with a converter like this:

public class DecimalFormatConverter : JsonConverter
{
    public override bool CanConvert(Type objectType)
    {
        return (objectType == typeof(decimal)
			|| objectType == typeof(decimal?));
    }

    public override void WriteJson(JsonWriter writer, object value, 
                                   JsonSerializer serializer)
    {
	writer.WriteValue(Convert.ToDecimal(value).ToString("G29"));
    }
	
	public override bool CanRead { get { return false; } }
	
    public override object ReadJson(JsonReader reader, Type objectType,
                                 object existingValue, JsonSerializer serializer)
    {
	throw new NotImplementedException();
    }
} 

All fine and dandy, except this now serializes into {"Value":"385"} which, again, produces incorrect hash of EC53BDEEC861E050E56FDA51B48621D0452006247D1501D79CF63A4C749E513F.

In order to return value as a numeric value and not string, you need to get a little bit more creative:

public class DecimalFormatConverter : JsonConverter
{
    public override bool CanConvert(Type objectType)
    {
        return (objectType == typeof(decimal)
			|| objectType == typeof(decimal?));
    }

    public override void WriteJson(JsonWriter writer, object value, 
                                   JsonSerializer serializer)
    {
	var valCasted = Convert.ToDecimal(value);
	if (Math.Round(valCasted, 10) == Math.Truncate(valCasted)) 
	{
		writer.WriteValue((int)Math.Truncate(valCasted));
	}
	else 
	{
        	writer.WriteValue(valCasted);
	}
    }
	
    public override bool CanRead { get { return false; } }
	
    public override object ReadJson(JsonReader reader, Type objectType,
                                 object existingValue, JsonSerializer serializer)
    {
	throw new NotImplementedException();
    }
} 

What this piece of code does is that, if decimal value, rounded to 10 decimal places equals to truncated decimal value, then it serializes the number as integer. Otherwise, it outputs it as it is.

But why 10 decimals? Sometimes a floating point error can make your value be 385.0000000000004. That is supposed to be 385, but remember what was said in chapter about decimal serialization in .NET? Serialization respects your data (well, almost) and keeps number of decimal spaces. So the value gets serialized as is: 385.0000000000004. Rounding a number to 10 decimal spots helps get rid of that.

This helped. Now, the serialized value in .NET is {"Value":385} and JavaScript serialization produces exact same result. Hence, both hashes are equal and response is considered valid.

Newtonsoft.Json package is probably one of the most essential packages in .NET software development. For those of you, not knowing what it does. It takes care of object serialization to JSON notation and deserilaization from JSON notation. I have used this package in numerous projects since its inception and I can only say great things about it.

However, as a part of one of our product’s GDPR compliance upgrade, I encountered an interesting undocumented feature. After object serialization a w3wp.exe process running the application pool for our product, started consuming 100% CPU capacity and hogged so much memory, we experienced “Out of memory exception” in a matter of minutes.

Since our product still uses .NET 3.5 (we are planning an upgrade to 4.7.2 shortly), tasks and parallel library are not native. We are using Microsoft’s TaskParallelLibrary package to circumvent framework deficiency. Hence, at first, I was dead sure that the library was a source of this issue. Specially, as we were doing serialization in an asynchronous method. After removing creation of new task, I was surprised it was not the case.

The object we wanted to serialize was a more complex derivative of this:

public class OurObject {

    #region Fields
    
    private int? _id;

    #endregion

    #region Properties

    public int Id {
        get {
            if (!_id.HasValue) throw new CustomException("some message");
            return _id;
        }
        set { _id = value; }
    }

    #endregion

}

The easiest way to serialize an instance of this object would be to do something like:

var instance = new OurObject();
var json = JsonConvert.Serialize(instance);

Except, this throws a CustomException, as Id property is not set. Newtonsoft.Json package documentation and StackOverflow answers offer a solution using serialization settings:

var instance = new OurObject(); 
var json = JsonConvert.Serialize(
    instance, 
    new JsonSerializerSettings { Error = (se, ev) => { ev.ErrorContext.Handled = true; } });

This works as expected. It ignores exceptions thrown by a serialized object instance. Yeay!

Not so fast. When using above code as part of web application, it will cause your application to hog all available CPU power and consume as much memory as possible. Promptly. Yikes! Surley, not something you would want in a production ready environment. Running a debugger revealed issues with this solution. Whenever a serialized object raised an exception, because of our serialization setting, exception went by unhandled. This in turn put stress on servers CPU and caused a memory leak of Mt. Everest.

The bad thing is, that at the point of my writing, there is no option, to tell JSON serialization engine to handle all exception raised and not just mark them as handled. I guess what you could do is create new properties for each property causing you a headache and then decorate it with JsonPropertyAttribute accordingly, but in our case, that would mean changing every property in an object (and there were plenty). What I ended up doing was that I converted an object to DataTable (we use it for ADO anyway) and serialized that. Worked like a charm.

Yesterday, I encountered an article on Medium titled A Better Bug Tracker by Anthony Sciamanna. The author goes to great lengths describing why bug trackers are unnecessary and point to the problem in your development workflow. Further, Mr. Sciamanna quotes Uncle Bob Martin (self proclaimed Software Craftsman):

 

“Think about what it means to use a bug tracking system. You have so many bugs you need an automated system to keep track of them.”

 

Now, far being from me, to disagree with such software developer authorities. And I do partially agree with points made in the article. You should have zero-bugs policy. Yes, you should modify your process to reduce the number of it. Yes, you should write unit tests. Yes, yes and yes. However, both gentlemen either do not know the purpose of bug trackers or they just pretend they do not in order to promote their ways. Personally, I am not sure which is worse.

First of all. Unit tests are not a solve-it-all tool. Yes, they present de-facto specifications for your code. Yes, they do make you think about possible edge-cases. Still, the test is only as good as the developer that made it. Now, I expect some will start waving at me with code coverage reports. I am sorry to tell you. I have seen a code coverage of 100% and unit tests that weren’t worth the electricity used to produce them.

Next, bug/issue trackers were made for people to log bugs, features, tasks etc that they cannot attend to at this very moment and I am pretty sure that there isn’t a single bug tracker out there that was made with an intention of encouraging developers to produce bugs.

Every developer I know, keeps some sort of log for features that need to be implemented, bugs that need to be fixed and tasks that must be performed (either in Notepad++, Excel or JIRA) and I am pretty sure the author of said article as well. The question is why do we, developers, log bugs? The answer is simple. So they don’t get lost or forgotten. Yes, I get fix bugs-first policy, but let’s say you are just fixing a bug and now a new bug report gets in. Should I stop fixing the bug I am currently working on? No. I will log the new one and continue working on my existing work.

I am glad to hear that Mr. Sciamanna and Uncle Bob Martin can hold everything they have in a queue in their head while doing continuous context-switching (or maybe, they are just not that busy). I am, sadly, not of that sort. If you tell me two things at once while I am doing something completely different, you will be lucky, if I fully remember one. Hence, I tend to write things down. And here is where bug tracker comes in handy. I use it to log ideas for new features, bugs, tasks that await me during the day, the full Monty. Sure, you can use excel spreadsheet for that, but doesn’t that spreadsheet then becomes a simple bug tracker?

Not using a bug tracker does not imply that your software doesn’t have bugs. Much like sticking your head in the sand doesn’t make your rear end invisible to innocent observers on land. It makes you look stupid, though.