Tuning Java Garbage Collection for VoltDB
As part of the command logging work I am revisiting latency in VoltDB and trying to make it more consistent.One of the things I found was that with a 512mb heap that is the default for the voter app...
View ArticleOptimal VoltDB hardware config for synchronous low latency command logging
If you want zero exposure to data loss when using command logging then you need to enable synchronous command logging. If you want to get consistent low latency with synchronous command logging then...
View Articlefsync durability
Kudos to Brad Fitzpatrick for his very useful fsync tester. I ran it on my desktop SATA, a battery backed controller, and on an SSD.On my desktop I was able to do about 700 ops with no write barriers...
View ArticleBulk deletes in VoltDB
Right now deletes are a little tricky to make performant, but it is possible.The current storage engine implementation does not allow external fragmentation at the expense of space (and for deletes...
View ArticleBatch processing VoltDB Snapshots
We have a utility for converting a VoltDB snapshot to a CSV, but that seems like a waste of time when you can read the files directly. org.voltdb.sysprocs.saverestore.TableSaveFile will open a snapshot...
View ArticleAnother snapshot processing task
A user wanted to know if the replicas of a partition had diverged. An easy way to check they at least have the same rows is to extract the primary keys, sort them, and then write them to a file. If you...
View ArticleDon't suffer in silence
At this point I have encountered several users who have had nodes fail, but never told anyone.Please don't do that! It doesn't matter what you were doing, whether this was a production cluster, or...
View Articleext3/ext4 data mount option
I did a quick test today to find out what data=[writeback | ordered] does to the performance of appending to a file vs. preallocating the file. This is on a 3.5" SATA disk. barrier=1,noatime,nodiratime...
View ArticleSnappy compression in VoltDB
John Hugg recently brought Snappy Java into Volt for use in the WAN replication feature that is under development. I did some quick benchmarks and I was really impressed with the performance in...
View ArticleUserspace distributed transactions in VoltDB
This post is best described as beta. I will update it based on the questions and confusion it generates.ImpetusAround Thanksgiving I came across a performance graph from a user with a workload that...
View ArticleDistributed system reliability
I wanted to briefly +1 a blog post by Jay Kreps.Architectures that emphasizes resilience around node and network failures do nothing to save you from software bugs and cascading failures. Based on my...
View ArticleVolt core and Initiation V2
Things have been pretty quiet for a while because we have been furiously working on what amounts to a rewrite of 1/3rd or so of Volt. The list of things we want to fix, but could not seem to improve...
View ArticleOverflow to disk
One of the things being talked about is how to leverage disk to expand the capacity of Volt. To make this work we have to incorporate disk bound data without slowing down the rate at which in memory...
View ArticleJitCask
I started working on JitCask, a pure Java clone of Basho's BitCask, this weekend. One of the main strengths of BitCask's design is its simplicity. I was able to put together an implementation in < 8...
View ArticleLeap seconds
Over the weekend Java processes everywhere started going haywire due to the leap second that occured. When I got to work Eclipse on my desktop was @ %550 utilization doing nothing, and when I killed...
View ArticleScale up at last
IV2 is finally far enough along to measure. I removed the last set of global locks around admission control and request routing today and did some tests on EC2 cluster compute instances. Ryan and Izzy...
View ArticleEC2 low TCP throughput
I've never been a fan of EC2. It's expensive, has highly variable performance, and adds enough complexity to cause reduced availability requiring you to over provision and over replicate across...
View Article520µs
Part of the motivation for IV2 was to reduce latency so that people could write client applications that use the traditional thread per request model that doesn't provide very high concurrency.I did a...
View ArticleGreat LKML thread from 2k3
I was looking into how to avoid partial page writes with Linux's buffer IO for VoltDB command logging. I noticed that there is still some read IO which I suspect is read ahead for partial page writes....
View Article10-gig E per socket limits and replication
Note that for this post socket always refers to TCP/UDP sockets and not a physical CPU socket.Chasing 10-gig E with Cluster Compute instancesMost of my testing has been on EC2 cluster compute instances...
View Article
