Sequence Diagram for SizedTieredCompaction Strategy in Cassandra

SizedTieredCompaction in Cassandra

Compaction in Cassandra:

CompactionManager.java : submitBackground(ColumnFamilyStore cfs) is called from CassadraDaemon.java .
The operation is done on per ColumnFamily basis, where cfs s reference to ColumnFamilyStore
This method checks for the count of column families in the columnFamilyStore(compactingCF) which is an instance for com.google.common.collect.ConcurrentHashMultiset of type Multiset<ColumnFamilyStore>(a google API for multisets where multisets a grey area between List and Seti.e. allows duplicate entries for Objects(feature from List) but No Order gauranteed (from Set)).
CompactionExecutor.java is used as executor that extends ThreadPoolExecutor to execute all compaction Tasks.
List<Future<?>> futures = new ArrayList<Future<?>>() is used to store all the future tasks to be executed as
executor.submit(new BackgroundCompactionTask(cfs)) until currently executing threads(activeCount) and size of futures List < maximum allowed threads(MaxPoolSize).


BackgroundCompactionTask is a Thread with run() method invoking
    a). the CompactionStrategy to be implemented
    b). to get the nextBackgroundTask having gc_before as argument


Compaction Strategy: The strategy is implemented as WrappingCompactionStrategy which extends AbstractCompactionStrategy, where in AbstractCompactionStrategy calls validateOptions(options) where the following options are validated:
TOMBSTONE_THRESHOLD_OPTION
TOMBSTONE_COMPACTION_INTERVAL_OPTION

The Following methods are called further:
 reloadCompactionStrategy(cfs.metadata);
 cfs.getDataTracker().subscribe(this);

getDefaultGcBefore(ColumnFamilyStore cfs)
Default CompactionStrategy is SizedTieredCompaction Strategy whose method getNextBackgroundSSTables(gcBefore) is being called.

   Iterable<SSTableReader> candidates = filterSuspectSSTables(Sets.intersection(cfs.getUncompactingSSTables(), sstables));
        candidates = filterColdSSTables(Lists.newArrayList(candidates), options.coldReadsToOmit);

filterColdSSTables() sorts the sstables on basis of the hotness(coldest fitrst) which in turn is the reads per sec per key for a particular sstable determined by RestorableMeter.java and is returned as the Map of sstable and the corresponding hotness value. This method removes as many cold sstables as possible.  and returns a list of SSTableReader.java
This List returned is  a possible sstables eligible to come under forthcoming Compaction process as candidates

createSSTableAndLengthPairs(candidates) is called to get a list of pairs with each possible candidate  (sstable) and the sstable size on disk as pairs.

Disk Size is determined using the SegmentedFile.java which uses the mmap(Memory Mapped File ) I/O java.nio package

After getting the pairs of the possible candidates as sstable and sstable length on disk, the files of similar size are suppose dto be grouped into buckets. This is accompalised by method getBuckets()

getBuckets() sorts the pairs of candidates as pair.right means, sorted on basis of the length on disk.
Iterates the sortedFIles and place the sstables sizeon disk  as key and  List of all sstables falling within the ranges as determined by bucketHigh, bucketLow and minSSTableSize as value.

The list of buckets, List<SSTableReader>  are further set to be categorized as most interesting means to filter out the buckets which meet the criteria to qualify as the minThreshold and maxThreshold.
case 1:
most intresting buckets re determined as the max(prunedBucketsAndHotness, bucketsByHotnessComparator)
case 2:
 In case no standard sstables are eleigible for compaction than,
checks for the tombstoneCompactionInterval which is the time elapsed between the sstable creation time and currenttime
so if,
Cassandra tracks tombstone droppable time for all TTLed/deleted columns and performs standalone compaction onto an SSTable that has droppable tombstones ratio against all columns above certain threshold.

google Sets API: Intersection and Union

import com.google.common.collect.Sets;

import java.util.HashSet;
import java.util.Set;

/** * Created by akhil on 1/26/2015. */public class SetsExample {


    public static void main(String args[]){

        Set<String> firstSet = new HashSet<String>();

        Set<String> secSet = new HashSet<String>();


        firstSet.add("akkhil");

        firstSet.add("kumar");

        firstSet.add("gupta");



        secSet.add("kapil");
        secSet.add("gupta");


        System.out.println("Interacion Example: Common items  in both sets"+ Sets.intersection(firstSet,secSet));

        System.out.println("Union Example: Total items  in both sets"+ Sets.union(firstSet,secSet));

    }



}

use of com.google.common.collect.ConcurrentHashMultiset; a gray area between List and Map means, allows duplicates(from list)+ but no ordering gaurantee(from Set)

import com.google.common.collect.ConcurrentHashMultiset;
import com.google.common.collect.Multiset;

/** * Created by akhil on 1/26/2015. */public class MultiSetExample {

    public static void main(String args[]){


        Multiset<String> test = ConcurrentHashMultiset.create();

        test.add("akkhil");

        test.add("gupta");

        test.add("akkhil");

        test.add("akkhil");


        System.out.println("Number of occurances for object "+test.count("akkhil"));



    }


}