by ·0 Comments

1. Tendis

From Tencent, based on RocksDB as persistent storage. Binlog to support resuming transport when running replication.

2. Kvrocks

Apache project, poor document. Have seen no advantage over Tendis so far. Also based on RocksDB.

3. KeyDB

Redis fork, multi-threading adopted.

4. Codis

A Redis proxy, clients are not required to know the cluster protocol.

5. redis-cluster-proxy

A Redis proxy, clients are not required to know the cluster protocol.

6. RedisShake

Redis data synchronization(Cross DC). Launch multiple process if sync source is a cluster setup. Used for one-shot full sync scenario, not recommanded for long-time incremental sync.

by ·0 Comments

Actually a simple note to the previous article.

– C: libevent seems to give best performance, but also low level.
– C++: drogon has websocket support, but no sse. async.
– C++: poco is sync.
– C++: workflow is low level, hard to use. wfrest has convenient apis, sse support in trunk.
– C++: boost/beast has poor performance, and cannot utilize multi-core cpu.
– C++: oatpp is async, but complex framework.
– Better use high level languages like Java or Go if running a web application.

by ·0 Comments

Read documents of Apache shardingsphere several years ago, and used to think it is the best database sharding library in client side. After trying to use it in a real-world application, problems reveal. First, the ecosystem has grown so large. Even a demo spring boot application can reference lots of dependencies. Second, when loading large data set from multiple shards, multi-threading is not used. I still have to manually implement it myself to improve load time.

Actually, what I need is the ability for selecting a database shard implicitly. When I write select t_user from..., it is rewritten to select t_user[0-7] from.... Here’s some alternative options I found:

1. hibernate interceptor

Refer to javadoc of StatementInspector class.

2. datasource proxy

See: https://jdbc-observations.github.io/datasource-proxy/docs/current/user-guide/#built-in-support

3. spring boot 3

See: https://spring.io/blog/2022/05/02/ever-wanted-to-rewrite-a-query-in-spring-data-jpa

But spring boot 3 requires java 17 and it only applies to jpa repository.

by ·0 Comments

Verified on CentOS7 and Windows 10.

1. Install v2ray and run proxy

v2ray unblocks github access from mainland China. Install v2ray clients and set IE proxy _only_ on Windows, bootstrap.bat & vcpkg.exe picks it automatically.

2. Download vcpkg from github and bootstrap

Download from: https://github.com/microsoft/vcpkg/releases

Export vcpkg-2022.08.15 directory as ${VCPKG_ROOT}.

3. Install drogon framework for demo

The drogon framework is a high performance application framework, including client & server supports. vcpkg builds static(*.a) library by default, use x64-linux-dynamic for dynamic(*.so) library. The repo version requires g++-8 to build, install from CentOS SCL:

To build with g++-7, manually install boost-filesystem package in vcpkg, and edit ${VCPKG_ROOT}/ports/drogon/portfile.cmake and comment out:

On Windows, open the command line for Visual Studio develop environment.

If openssl build fails, run:

If other errors, try to update to recent github ports. In my case, libmariadb build failed, that have been fixed in master.

4. Export drogon framework

5. Add a demo program

Linux dynamic build is community supported, invoke cmake with:

Now build with make or Visual Studio.

6. Stick to a specific version

add a vcpkg.json file:

It sticks to drogon 1.8.0 and openssl 1.1.1n. ${VCPKG_ROOT} now required to be a git repository. In your project directory, install specific versions of libraries by running:

Run cmake:

Now ldd output shows openssl 1.1 (default build is 3.0):

The only difference is the existence of vcpkg.json file, when using versioning.

7. Binary caching

If you change the root path of vcpkg, better clean up the cache, or build may fail. It’s $HOME/.cache/vcpkg/archives under Linux, and %LOCALAPPDATA%\vcpkg\archives under Windows.

by ·0 Comments

Just got time to investigate and summarize them. Actually I started from the Microservices book, it introduced Saga & transactional outbox pattern. The sample code is too complex, abstracted too well, and hard to trace. After finished reading the book, I still have no option about how to deploy disturbed transactions to my services, without the author’s framework. And many details are messed up for me. For instance, what’s the difference between Saga, TCC & XA? How to handle partial failures? What if it fails in rollback/cancel phase? How about the latency of Saga? the consistency? I cannot find the precise answer, it is just a book for students, not for engineers.

I investigated several frameworks:

1. Seata

Seata/Alibaba is the most famous one. It runs in XA, Saga, TCC, and an additional AT(Auto Transaction?) mode. AT mode is actually an application-level XA, which is business-agnostic. Seata manages and generates rollback SQLs for you, but there are limitations. I am most interested in TCC, which is easy to understand, and is the most complete solution to distributed transactions. Also, it requires most effort to implement. From the recent release notes, there are still fundamental bugs, and documents are hard to read, which make me wonder its usability.

2. Hmily

Hmily(How Much I Love You)/JD is the second one, since it’s in Java. From the point of an end user, it very easy to integrate. It provides annotations to simplify TCC implementation. But one critical design bug(via quick scan of its code): It saves its transaction logs asynchronously via disruptor, which makes it much easier to lost transactions and lead to an inconsistent status.

3. DTM

dtm/(Bytedance? Tencent?) should be one with most potentialities so far. It clears most of my uncertainty. It has most informative documents, and helps me to choose between all distributed transaction modes. When talking about consistency: XA > TCC > 2-phase message > Saga. And Saga is most useful in long transactions.

One innovation(and patent?): it introduced subtransaction barriers. The mechanism perfectly handles repeated request, dangling action, dangling compensation in TCC automatically, without user attention.

And the only drawback is: it is written in golang(I even learned go programming meanwhile). Hopefully, it provides lightweight restful APIs.

4. ByteCC

ByteCC. Not investigated. Seems not actively maintained.

5. EasyTransaction

EasyTransaction. Not investigated, here is a review(In Chinese) from the author.