We have experienced difficulties in both these areas using the Entity Framework, and in this article, I will be describing the cause of the issues, and the solutions that we at MetaSys came up with. If you are having performance issues with Entity Framework, this article might provide some useful insight and suggestions.

Our issues with the  Entity Framework:

Let me quickly brief you about how we started using Entity Framework. We started application development of a .NET web application roughly 10 years ago. At the time, Entity Framework was a new concept introduced by Microsoft. Its purpose was to allow the developer to more easily write SQL queries including calculations, it also simplified CRUD operations and handled results into objects. We used Entity Framework for our web application, and during initial testing, everything was working very well.

The performance issues arose after the client started using the application, particularly as the amount of data in the database started growing. We used the Ants Profiler tool to identify the root cause of poor performance. It showed us that stored procedures were executed fast without any significant delay, but with the Entity Framework code, it was taking a long time to render data on a page.

Another issue was that the SQL database for the application had more than 300 tables. Updating the Entity model with a change in any of the tables would take a very long time. It was also difficult to merge changes of only one developer, or only one module, as it would update the entire Entity model. This made it a challenge to release the application module-wise.

MetaSys Approach :

To overcome performance issues, we first tried to change some of the settings of EDMX, and secondly updated the Entity Framework to the latest version. Neither made much difference to the performance. In the meanwhile, the applications database size and complexity kept on growing, as the application grew.

Eventually, we replaced the Entity with ADO.NET, and we immediately saw a significant improvement in performance. The difficulty we faced with the conversion was how to handle the ADO.NET result into objects. We resolved this using the open-source Dapper ORM. Dapper is a framework for mapping relational object models. Like Entity Framework, It eases the handling of data in objects and supports multiple data query results. This solution not only improved the page loading time, but as there was no need to update the entity model, the developer’s time also reduced significantly.

So far we have found that using ADO.Net with Dapper ORM solved all the problems we experienced with the Entity Framework.

About Us:

Our team of Dot Net developers have experience of building Dot Net solutions using Microsoft technologies for more than two decades using VB to latest .Net Core applications. For more info. https://www.metasyssoftware.com/dot-net

This article details such an integration project. Our Dot Net application team helped a client integrate InBody technology into their application. 

InBody devices offer detailed measurement of body composition balance, including key health factors such as protein, minerals, BMI and body fat percentage. Our project revolved around integrating the InBody 570 device with a .Net application. The solution we implemented involved the .Net application listening to the serial ports attached to the device for the duration of the test. After receiving a data stream from the InBody device, relevant data is extracted using indices of specific data factors mentioned in the InBody 570 technical documentation. Key settings required by the .Net application include baud rate, parity, data bits, and stop bits. The InBody technical specification includes more details about the values of these settings.  Once the data stream is received, and the data factors are extracted, the data was saved in an SQL server database. The .Net web application reads this data from the SQL server database and displays it to the user. 

Key Challenges

The idea is that the application can be used by the fitness facility subscribing to the application and having an InBody machine. The application runs on a laptop or desktop computer connected to the Inbody machine. However, one challenge was that there is no guarantee that the computer is always connected using the same port. If the computer had more than one port on which the device stream can be received, then we had to lock and keep listening to all available ports. Our solution to these issues was to program the app to lock and keep listening to all available and open ports of the laptop and as soon as we receive either data stream or exception, we unlock all ports locked by the .net application. It is important to note that only that process which has locked the serial port can unlock it, no other process can forcibly remove that lock. In the event that the process which locked the port gets terminated, one is left with only the option of system reboot in order to unlock that port.

Another issue is that the same app was required to integrate with another measurement device manufactured by InBody, the BPBIO 320. This device reads systolic blood pressure, diastolic blood pressure and pulse, and we modified the .Net app to work with both InBody devices. We used a condition-based code, which looks at application settings and accordingly saves the data in either the production or the staging environment. The BPBIO 320 requires a different baud rate to that of InBody 570, therefore we adapted the application so that the user enters the test type, and then the baud rate is automatically set up by the software. We also process information contained within the data stream that indicates status such as “Measurement started”, “Measurement interrupted due to use of the stop button”, and “Measurement interrupted due to error”.  These cases are handled in the application and clear information is passed on to the user.  

We used a PuTTY tool to simulate the device stream in the development environment, as the physical device was not always available. Device testing was performed after development was complete, and we made the necessary changes to the application before the launch.

We will be glad to help anyone interested in doing similar kinds of integrations in their software. For more info https://www.metasyssoftware.com/case-study-dotnet

Improving database performance is a challenging task that developers need to pay attention to in order to reduce the response time for the users. In this article, I am sharing a technique for Database Performance Optimization using Data Partitioning.

What is Data Partitioning?

Data Partitioning refers to the database process of dividing large tables into smaller parts by  Rows and Columns. A major benefit of Data Partitioning is that one only needs to specify the partitioning criteria, and then SQL handles the data by itself.

SQL uses multiple .ndf files (Secondary Database Files), in addition to the .mdf file (Primary Database File) to divide and store data based on the partition criteria. The user needs to specify those .ndf files and provide the partition criteria by using Partition Schemes and Functions. SQL Server then takes care of data maintenance by itself.

Data Partitioning can be divided into two categories:

1. Vertical Partitioning: In vertical partitioning, data is divided column-wise. For example, if  a database contains a table with 10 columns of which only 5 columns are used frequently, it can be divided into two tables with 5 columns each. This way a table is made more efficient, by dividing it into multiple filegroups based on columns.
2. Horizontal Partitioning: In horizontal partitioning, data is divided row-wise. For example, suppose there is a table with 1 lakh of records for the period 2011 – 2019, where each record includes a date. Quite often it is only necessary to access the records for last year, i.e. 2019 or recent years -2017,2018. Through horizontal partitioning, the data can be divided into yearly records in multiple filegroups.

By using these Data Partitioning techniques, we can enhance the performance of our database. This is a highly recommended technique for applications where Reporting is frequently used.

Worked example:

We have a table called tbl Data Partitioning with approximately 2.5 crore records. The data includes records from 2011 to 2014, of which the 2014 records are 45 lakhs. The Table looks like:

To execute a select query to find the 2014 records takes almost 23 seconds.

Using Horizontal Data Partitioning on the table using Created Date, the time can be significantly reduced.

For that, we first need to specify the filegroups:

After creating Partition Functions and Schemes, we can map the partitions to filegroups.

Finally, After implementing Data partitioning, the time taken by the Select query is reduced from 23 seconds to 15 seconds:

Advantages of Data Partitioning:

1. Improves performance for larger tables.
2. Fast and expensive storage can be used for active data filegroups, while keeping the less active data filegroups on slower/less expensive storage.
3. Partial backup and restoration is possible.
4. Can be used temporarily as a substitute for data archiving.

Our team has experience of working on PHP MySQL development projects and has handled many assignments from small informational website to robust enterprise applications. For more information https://www.metasyssoftware.com/php-mysql