Is it Really Durable?

2009/11/11 11:45
mikio

I read an article about Tokyo Tyrant and found it intersting. As it says, calling sync operation periodically is useful to improve durability though it sacrifices performance.

However, at least in the typical usecase of TT, that strategy is not suitable. I work at mixi.jp and we operate large databases where the writing throughput is more than 10,000 QPS. So, we use TT in some situations instead of MySQL. We know that the tricks like calling sync to improve durability in each local machine is meaningless.

In mixi.jp, the most frequent cause of the database crash is by hardware breakdown, especially in hard disks and their RAID controller. So, even if we called sync every update operation, no data would be recovered at each local machine. They sometimes can't be booted up any longer.

Yes, we use replication. Every database server has at least one slave database server. That is, all records are replicated among at least two servers. So, if one server crashes, we drop the server out of the service and add a new alternative server. Records of the new server are repaired with the backup database file and difference between the backup time and the current time is solved with the update log of the surviving server by the replication mechanism.

Insistently I say, please use replication, don't believe in your hardwares. Calling sync is just a placebo in actual usecases.


New Site Has Been Opened

2009/09/01 10:54
mikio

Though I have used SourceForge.net to release my open source products since 2003, eventually I abandoned to use it. I don't like the recent renewal of the user interface. It's difficult to find developer functions and user forums. The shell service and the complie farm were abolished precedently. Now, there's no attractive feature on SF.net for me. I'm very thankful to SF.net because I think that their contribution to open source communities is awesome. A great number of open source products including mine have been hosted there. However, I desided to open my own site and move my products from SF.net to here.

This site is on VPS (virtual private server) and I can be the root user. Despite of the functionality, it costs only 980 yen (about 10 dollers) a month. However, I feel it is not stable. I have experienced unexpected crash three times already. Hmm, I should claim technical support to the administrator.

BTW, I'll describe features and use cases of Tokyo Products and my other products here. As I am not so good at English that I can't understand complex questions from users, I want to improve my skills by practicing English here.