2022/02/17

What is Encryption in Delphi?

 


I remember that when I mentioned that Delphi did not officially have an encryption unit in a discussion with my friends, I always thought it was incredible, because DataSnap has this encryption filter:

TTransportFilterItem.FilterId: PC1 / RSA

Since there is an encryption setting, the encryption method must be hidden somewhere in the Source paths.









It cost me 8 damn nights!



How exactly does DataSnap operate encryption? I muttered to myself as I silently stared at the screen in the dark.

Quest

Find of the DSTCPServerTransport unit, I saw the dependent DBXTransport, and I saw a sensitive word in "uses":

DBXEncryption

As if seeing a glimmer of light, I find the Delphi installation path and opened the mysterious DBXEncryption source code.

It begins with this:

PC1 encryption definitely came from here.

TPC1Cypher is the PC1 encryption class, and my intuition tells me that RSA is maybe TRSACypher.

But there is no trace of it in the DBXEncryption unit, is it a mistake? I try to find possible clues in the source code sea.

DBXOpenSSL

The beginning of the file reads like this :

Sure enough, as I expected, the RSA encryption method is named after [TRSACypher].

PC1 encryption filter provided by DataSnap Server

DataSnap Server provides PC1 encryption filter. At the beginning, everything uses the official default value. There is no problem in operation, but the Key value is fixed, so let's modify it.

After modifying, the operation is also success. After the normal measurement, I try the [what error will pop up if the server and client passwords are different].

The result can still run successfully!

Why does it work with different KEY values on both sides? After being tested, it goes through several units:

  • DBXTransport
  • DBXEncryption

There are Cypher / Decypher encryption and decryption functions in DBXEncryption.TPC1Cypher class. After testing, we know that when each pair of packets is encrypted and passed to the other party, it will also be passed to the other party together with the "KEY", and the other party will use the "KEY" to decrypt the packet, and vice versa Of course.

To put it simply, the packet can be decrypted correctly only when the KEY is used with the [same] algorithm to decrypt the packet.

 

Therefore, it is the key to know the correct algorithm. Although as long as it is a Delphi application, it can be unraveled, but in such a niche as Delphi, it can be regarded as a good encryption method.  😁


See also

2022/02/16

Memory Table Showdown! FdMemTable vs VirtualTable vs ClientDataSet, Part 1: Increase Data

Content

English Version

Recently, I saw people debating in a technical group about which method is better for writing data to a database, and it surprised me that the way a program controls writing to a database can affect the writing results. This piqued my curiosity, so I decided to conduct a test myself to see how much of an impact poorly written code could have on rapid memory table operations.

In Delphi's TDataSet, there are two well-known methods for adding new records:
  1. INSERT
  2. APPEND

What is "INSERT"?

Inserts a new, empty record in the dataset. 

What is "APPEND"?

Adds a new, empty record to the end of the dataset.

Some people say that INSERT is more efficient than APPEND, which is quite surprising. My previous impression was that APPEND was faster, but I don't remember how much faster it was. Now, I will write down the test results to see how much difference there really is between the two methods in terms of performance.

Testing method:

  • Test time: 100,000 records
  • Table structure: 3 fields (Integer, String, String)
  • Record unit: milliseconds

    Example using ClientDataSet

    From the above figure, it can be seen that Insert takes much more time than Append.

    Next is the time required for each ten thousand records:

    As the data amount increases, the time required for Insert also increases. When it reaches 90,000 to 100,000 records, it takes 797 ms.

    The result is obvious, Append is better.

    But this result is really boring!

    ClientDataSet is a type of Memory Table. If we compare it with other Memory Tables...

    Speaking of Memory Tables, there are many options to choose from. Here are the competitors for this test:

    • FireDAC -- TFdMemTable
    • Devart -- TVirtualTable
    • DataSnap -- TClientDataSet

    FireDAC -- TFdMemTable


    Similar to ClientDataSet, FdMemTable also spends more time on Insert than on Append.

    Devart -- VirtualTable





    In VirtualTable, we can also see that Insert takes longer than Append. But please note that the total difference of 32 ms is not very noticeable in terms of user experience. This means that users of VirtualTable do not need to pay too much attention to program details 😋.

    Three-way showdown

    The contestants have all completed their individual events, and now it's time for the showdown:




    Summary

    Although memory operations are very fast, careless handling can still lead to performance tragedies.

    In cases of massive data insertion, using the Append method is undoubtedly the best practice.

    The above conclusion is an exception for VirtualTable. 😁

    The performance difference between Insert and Append is the biggest in ClientDataSet. Developers who use it should pay attention to optimizing their code.

    FdMemTable, as the successor of ClientDataSet, is not good enough for performance-obsessed developers yet, but it still has room for improvement.

    "DBNavigator.Insert" and "TDataSetInsert" in ActionList are both "Insert" actions and neither of them has "Append". If needed, manual modification is required. Fortunately, in the case of "end-users", it will not have too much impact.


    Next time someone tells me that Insert has better performance than Append, I'll kick them out! 😊


    中文版本 (2017/08/14)

    最近在技術群組裡看到有人在爭論寫入資料庫的方式孰優孰劣,看到程式控制寫入資料庫的方法,居然也會影響寫入結果;這引發了我的好奇心,就自己來測試看看在飛快的記憶體表格操作,如果程式寫得不好,會有多大的影響。

    Delphi 的 TDataSet 在新增上有兩個著名的操作方法:

    • INSERT
    • APPEND

    What is "INSERT"?

    Inserts a new, empty record in the dataset. 

    What is "APPEND"?

    Adds a new, empty record to the end of the dataset.

    這兩者的比較,居然有人說 INSERT 比 APPEND 的效能要來得好,真是意外,我以前的印象中,記得是 APPEND 要來得快,但已經不記得快多少,現在就來把測試結果寫下來,看這兩者到底的效能差別到底有多大。

    測試方式:

    • Test time:100000 Record
    • Table Structure: Integer, String, String; 3 Fields
    • Record unit: ms

    以 ClientDataSet 為例:

    由上圖可知, Insert 遠比 Append 要花更多的時間。

    接著是每累加一萬筆時所需的時間:


    隨著資料量越多,Insert 所需的時間就越長,當九萬至十萬筆時,就需要花上 797 ms 的時間。

    結果很明顯,Append 勝出。






































    但這樣結果實在是太無趣了!

    ClientDataSet是一種 Memory Table,如果把其它的 Memory Table 一起抓來比較的話……

    說到 Memory Table 的選擇非常多,以下是本次參戰的選手:

    • FireDAC -- TFdMemTable
    • Devart -- TVirtualTable
    • DataSnap -- TClientDataSet

    FireDAC -- TFdMemTable



    就像 ClientDataSet 一樣,FdMemTable 在 Insert 明顯比 Append 花更多的時間。

    Devart -- VirtualTable






    VirtualTable 也可以看到 Insert 也要比 Append 要久。

    但請注意!32 ms 的總差距在體感上其實是差不多的!

    *意謂著使用 VirtualTable 的人可以不用太在意程式細節 (笑)


    大亂鬥

    選手們都已經完成個人賽事,接下來就要登台對決:




    總結

    僅管記憶體非常快,但操作不慎仍有可能造成效能悲劇。

    在大量新增的場合,使用 Append 的方式絕對是最佳作法。

    以上總結在 VirtualTable 是個例外。(笑)

    ClientDataSet 在 Insert / Append 效能落差最大,有在使用它的人要注意手上的程式碼優化工作。

    FdMemTable 作為未來 ClientDataSet 的接班人,對效能偏執狂的開發者來說還不夠好,請再繼續加油。

    「DBNavigator.Insert」和 ActionList 裡的「TDataSetInsert」兩者都是【Insert】行為,而且它們都沒有【Append】,需要的話則須人工修改。所幸在【一般使用者】的場合下,不會有太大的影響。






































    下次誰再跟我說 Insert 效能比 Append 好就踹飛他!



    See also: