2022/02/17

What is Encryption in Delphi?

 


I remember that when I mentioned that Delphi did not officially have an encryption unit in a discussion with my friends, I always thought it was incredible, because DataSnap has this encryption filter:

TTransportFilterItem.FilterId: PC1 / RSA

Since there is an encryption setting, the encryption method must be hidden somewhere in the Source paths.









It cost me 8 damn nights!



How exactly does DataSnap operate encryption? I muttered to myself as I silently stared at the screen in the dark.

Quest

Find of the DSTCPServerTransport unit, I saw the dependent DBXTransport, and I saw a sensitive word in "uses":

DBXEncryption

As if seeing a glimmer of light, I find the Delphi installation path and opened the mysterious DBXEncryption source code.

It begins with this:

PC1 encryption definitely came from here.

TPC1Cypher is the PC1 encryption class, and my intuition tells me that RSA is maybe TRSACypher.

But there is no trace of it in the DBXEncryption unit, is it a mistake? I try to find possible clues in the source code sea.

DBXOpenSSL

The beginning of the file reads like this :

Sure enough, as I expected, the RSA encryption method is named after [TRSACypher].

PC1 encryption filter provided by DataSnap Server

DataSnap Server provides PC1 encryption filter. At the beginning, everything uses the official default value. There is no problem in operation, but the Key value is fixed, so let's modify it.

After modifying, the operation is also success. After the normal measurement, I try the [what error will pop up if the server and client passwords are different].

The result can still run successfully!

Why does it work with different KEY values on both sides? After being tested, it goes through several units:

  • DBXTransport
  • DBXEncryption

There are Cypher / Decypher encryption and decryption functions in DBXEncryption.TPC1Cypher class. After testing, we know that when each pair of packets is encrypted and passed to the other party, it will also be passed to the other party together with the "KEY", and the other party will use the "KEY" to decrypt the packet, and vice versa Of course.

To put it simply, the packet can be decrypted correctly only when the KEY is used with the [same] algorithm to decrypt the packet.

 

Therefore, it is the key to know the correct algorithm. Although as long as it is a Delphi application, it can be unraveled, but in such a niche as Delphi, it can be regarded as a good encryption method.  😁


See also

2022/02/16

Memory Table Duel! FdMemTable vs VirtualTable vs ClientDataSet, Part 1: Increase Data

Content

English Version

Recently in the technical group, I saw some people arguing about the method of writing to the database. I saw that the method of the program controlling the writing to the database would actually affect the writing result. This aroused my curiosity, so I just Test it yourself to see how much impact it will have on the fast memory table operation if the program is poorly written.

Control methods about Addition data on TDataSet in Delphi

Delphi's TDataSet has two well-known operation methods in addition:

  1. INSERT
  2. APPEND

What is "INSERT"?

Inserts a new, empty record in the dataset. 

What is "APPEND"?

Adds a new, empty record to the end of the dataset.

Comparing the two, some people say that INSERT is better than APPEND. It's a surprise. In my previous impression, I remembered that APPEND was going to come faster, but I don't remember how much faster. Now I will write down the test results. Let's see how much the difference in performance between the two is.

How to test?

  • Test time:100000 Record
  • Table Structure: Integer, String, String; 3 Fields
  • Record unit: ms

Take ClientDataSet as an example

As can be seen from the above figure, Insert takes much more time than Append.

Next is the time required to accumulate insert 10,000 record:

As the amount of data increases, Insert takes longer. When 90,000 to 100,000 transactions, it takes 797 ms.

The result is clear, Append wins.

Got the result, but it's boring ! IF...

ClientDataSet is a kind of Memory Table. If you compare other Memory Tables together...

When it comes to the choice of Memory Table, the following are the players participating in this battle:

  • FireDAC -- TFdMemTable
  • Devart -- TVirtualTable
  • DataSnap -- TClientDataSet

FireDAC -- TFdMemTable


Just like ClientDataSet, TFdMemTable spends significantly more time in Insert than Append.

Devart -- VirtualTable






VirtualTable can also see that Insert also takes longer than Append.
Attention! The total difference of 32 ms is actually similar in physical sense!

Tips: Means people who use VirtualTable don't need to pay too much attention to programming details 😋

Super Smash Bros.

The players have all completed their individual events, and they are about to take the stage for a duel:




Summary

Even though the memory is very fast, inadvertent operation can still cause performance tragedy.

Using Append is definitely the best way to add records in a large number of cases.

The above summary is an exception in VirtualTable. 😁

ClientDataSet has the largest performance gap in Insert / Append, and those who use it should pay attention to the code optimization work at hand.

As the successor of ClientDataSet in the future, FdMemTable is not good enough for performance paranoid developers, please continue to work hard.

Both "DBNavigator.Insert" and "TDataSetInsert" in ActionList are [Insert] behaviors, and neither of them has [Append], which must be manually modified if necessary. Fortunately, in the case of [general users], it will not have much impact.


This is my very early test result. If you have different ideas, please share your experience. 😊


中文版本 (2017/08/14)

最近在技術群組裡看到有人在爭論寫入資料庫的方式孰優孰劣,看到程式控制寫入資料庫的方法,居然也會影響寫入結果;這引發了我的好奇心,就自己來測試看看在飛快的記憶體表格操作,如果程式寫得不好,會有多大的影響。

Delphi 的 TDataSet 在新增上有兩個著名的操作方法:

  • INSERT
  • APPEND

What is "INSERT"?

Inserts a new, empty record in the dataset. 

What is "APPEND"?

Adds a new, empty record to the end of the dataset.

這兩者的比較,居然有人說 INSERT 比 APPEND 的效能要來得好,真是意外,我以前的印象中,記得是 APPEND 要來得快,但已經不記得快多少,現在就來把測試結果寫下來,看這兩者到底的效能差別到底有多大。

測試方式:

  • Test time:100000 Record
  • Table Structure: Integer, String, String; 3 Fields
  • Record unit: ms

以 ClientDataSet 為例:

由上圖可知, Insert 遠比 Append 要花更多的時間。

接著是每累加一萬筆時所需的時間:


隨著資料量越多,Insert 所需的時間就越長,當九萬至十萬筆時,就需要花上 797 ms 的時間。

結果很明顯,Append 勝出。






































但這樣結果實在是太無趣了!

ClientDataSet是一種 Memory Table,如果把其它的 Memory Table 一起抓來比較的話……

說到 Memory Table 的選擇非常多,以下是本次參戰的選手:

  • FireDAC -- TFdMemTable
  • Devart -- TVirtualTable
  • DataSnap -- TClientDataSet

FireDAC -- TFdMemTable



就像 ClientDataSet 一樣,FdMemTable 在 Insert 明顯比 Append 花更多的時間。

Devart -- VirtualTable






VirtualTable 也可以看到 Insert 也要比 Append 要久。

但請注意!32 ms 的總差距在體感上其實是差不多的!

*意謂著使用 VirtualTable 的人可以不用太在意程式細節 (笑)


大亂鬥

選手們都已經完成個人賽事,接下來就要登台對決:




總結

僅管記憶體非常快,但操作不慎仍有可能造成效能悲劇。

在大量新增的場合,使用 Append 的方式絕對是最佳作法。

以上總結在 VirtualTable 是個例外。(笑)

ClientDataSet 在 Insert / Append 效能落差最大,有在使用它的人要注意手上的程式碼優化工作。

FdMemTable 作為未來 ClientDataSet 的接班人,對效能偏執狂的開發者來說還不夠好,請再繼續加油。

「DBNavigator.Insert」和 ActionList 裡的「TDataSetInsert」兩者都是【Insert】行為,而且它們都沒有【Append】,需要的話則須人工修改。所幸在【一般使用者】的場合下,不會有太大的影響。






































下次誰再跟我說 Insert 效能比 Append 好就踹飛他!



See also: