It has been a long time, since I have written a post. Reasons vary. Most of it is down to my laziness and limitations to my spare time. Some of it is down to lack of motivation as well.

Anyway, during several last months I have, surprisingly, learned many new things. I limited my pick to the following items:

  1. You cannot set Prefer 32-bit option to class library in .NET
  2. ORACLE RDBMS column names must not exceed 30 characters
  3. People, suggesting that copy & paste for VPN connections must be disabled, should be “taken care of”
  4. No matter what the task is, you must take your time to solve it
  5. CSRF feature, known as warning SG0016 is annoying, if you are implementing public API
  6. How to use query string parameters in non-RESTful API
  7. FastDirectoryEnumerator!
  8. When using integration to move some data, always use separate table

Now to details.

 

You cannot set Prefer 32-bit option class library in .NET

The setting can be located in Project properties -> Build, but it is disabled for class libraries. First of all, as per this StackOverflow article, the only difference between selecting “x86”  as platform target and using “Prefer 32-bit” option is that application compiled for “x86” will fail on ARM based environment, while application compiled for “Any CPU” with “Prefer 32-bit” selected will not. My reasoning is that as executable projects are meant to define the architecture for entire application, this setting would have no meaning in class libraries. Hence, it is disabled.

 

ORACLE RDBMS column names must not exceed 30 characters

Really. But only, if you are running version 12.1 or lower. Otherwise, you can use names up to 128 characters. We found that out the hard way, while migrating MSSQL database to ORACLE platform. Anyway, you can find out what length your column and table names can be, by running following statment in your SQL client:

describe all_tab_columns

 

People, suggesting that copy & paste for VPN connections must be disabled, should be “taken care of”

The title says it all really. Disabling copy & paste option over VPN connection might have some security benefits and I am pretty sure that some auditor can’t sleep, if it is not, but it is annoying as hell for anybody that actually tries to use VPN connection for REAL work. Imagine you have to prepare a report for a customer that requires you to run a 300 line long SQL statement. Obviously, you are not developing that in their environment. You are doing it in your local database. Now, you just need to somehow get it to the customer system. Copy & paste seems harmless enough. Yeah, not going to happen. So now, you need a Dropbox (best case scenario) or, mailing that SQL to customer’s admin and hoping that person knows what he/she is doing. Not to mention the awkward situation, when you find out you forgot to add just one more column or condition to your SQL statement.

Kudos to all auditors, recommending ban of copy & paste. NOT.

 

No matter what the task is, you must take your time to solve it

Sounds reasonable enough. Right? Except, when you are bogged down with work, and now a trivial, but urgent task comes in, forcing you to drop everything and focus on that specific task. Hah, but the task is trivial. What could possibly go wrong? Well, for starters the fact that assumption is a mother of all clusterfucks (pardon my French). So, now, you solved the task half-arsed, passed it back to customer, only to let it hit you right back on your head 30 minutes later. Instead of doing it properly, the first time, you will have to do it the second and hopefully not the third time, taking even more of the time you didn’t have in the first place. Meanwhile, your reputation with your customer is sinking faster than RMS Titanic.

Even in times of stress and distraught, it is important to remember, that each and every task is worth your attention. If naught else, it will save you minutes, if not hours and leave your reputation intact.

 

CSRF feature, known as warning SG0016 is annoying, if you are implementing public API

“New” Visual Studio 2017 comes with abundance of new features. One of them is giving you security recommendations that behave as warnings. Roslyn Security Guard it is called. All fine and dandy. Sadly, though, most of those recommendations are useful only, if you are developing an internal applications. If you are building, let’s say, public Web API, you really don’t want to hear about that CSRF SG0016 warning telling you to validate anti-foregery token. Specially, as all requests are coming from other servers and you have no way to validate that token.

There is a workaround to add

#pragma warning disable SG0016

just below class declaration, which suppresses the warning until you do this

#pragma warning restore SG0016

I would have still preferred a project option to disable that, though.

 

How to use query string parameters in non-RESTful API

I had to connect to 3rd party non-restful API, that invented all sorts of parameter passing options. From classic JSON for POST requests to combination of route parameters and query string parameters. As I had no access to the API from my development environment, I created a mock API and had to mimic the original API’s behavior.

For route parameters, you simply define a route that knows how to handle them, like so:

config.Routes.MapHttpRoute(
    name: "DefaultApi",
    routeTemplate: "api/{controller}/{id}/{taxId}",
    defaults: new { id = RouteParameter.Optional, taxId = RouteParameter.Optional }
);

If you want to obtain parameter from query string though, you must define [FromUri] in front of it in method declaration:

[HttpGet]
public HttpResponseMessage QueryCustomers([FromUri]string taxId)

 

FastDirectoryEnumerator!

A quick task. You need to move 10.000 files from one folder to another.

Solution 1

Use Directory.GetFiles to get a list of all files in directory and then use File.Copy to move them to another location.

Problem with this solution, however, is that although it works fast, it will store all file names into a string array, thus hogging your memory resources like crazy.

Solution 2

Use Directory.EnumerateFiles to get a list of all files in directory and then use File.Copy to move them to another location.

Much better solution as it returns files as IEnumerable<string> which allows you to traverse files before all are loaded.

 

Now imagine that source or destination or both for files that need transfer are on network drive. In that case, first solution will take around 30 seconds to read all files. Second will not fare much better, getting all files read in about 25 seconds. And this on a fast network drive.

Introducing FastDirectoryEnumerator for next solution.

Solution 3

Using FastDirectoryEnumerator.EnumerateFiles, it read 10.000 files in about 20 miliseconds. Yes, that is right. Miliseconds.

You can check documentation and implementation on CodeProject site. The secret is, apparently in not doing a round-trip to the network for each and every file. That and using kernel32.dll.

 

When using integration to move some data, always use separate table

Another project of mine has a bug. Yet to be decided, if it is human or code, but in any case, code should prevent such situations.

This is what happens. The code moves some data from table ITEMS via 3rd party web service to their product. This is done by a column named STATUS in the table ITEMS, which must hold a certain value. The code sets status to “moved to 3rd party service”, prior completion and to “error” in case of execution errors. Upon completion a 3rd party code is written into another field (let’s call it EXT_ID).

Unfortunately, web interface for adding and editing items also uses STATUS field for document workflow. Meaning, it sets status on certain actions.

Lately, this started to happen. An item gets picked, status is set to “moved to 3rd party service” and transfer completes and sets EXT_ID. During this process someone with item opened in browser clicks on “Confirm” button again in web interface and sets status back to “pending for transfer”. Action also removes EXT_ID. As 3rd party service checks for duplicates, it returns a duplication error.

To avoid this, a way better solution would be to create a table ITEMS_TRANSFER. The row would be added to this table (with hash of values), when transfer would be requested and removed (or marked as removed) when transfer completes. This would certainly prevent duplication errors.