Package org.apache.cassandra.db

Class ColumnFamilyStore

    • Field Detail

      • SNAPSHOT_TRUNCATE_PREFIX

        public static final java.lang.String SNAPSHOT_TRUNCATE_PREFIX
        See Also:
        Constant Field Values

      • SNAPSHOT_DROP_PREFIX

        public static final java.lang.String SNAPSHOT_DROP_PREFIX
        See Also:
        Constant Field Values

      • keyspace

        public final Keyspace keyspace

      • name

        public final java.lang.String name

      • readOrdering

        public final OpOrder readOrdering

      • viewManager

        public final TableViews viewManager

      • sampleReadLatencyNanos

        public volatile long sampleReadLatencyNanos

      • additionalWriteLatencyNanos

        public volatile long additionalWriteLatencyNanos

    • Constructor Detail

      • ColumnFamilyStore

        public ColumnFamilyStore​(Keyspace keyspace,
                         java.lang.String columnFamilyName,
                         int generation,
                         TableMetadataRef metadata,
                         Directories directories,
                         boolean loadSSTables,
                         boolean registerBookeeping,
                         boolean offline)

    • Method Detail

      • shutdownPostFlushExecutor

        public static void shutdownPostFlushExecutor()
                                      throws java.lang.InterruptedException
        Throws:
        java.lang.InterruptedException

      • shutdownExecutorsAndWait

        public static void shutdownExecutorsAndWait​(long timeout,
                                            java.util.concurrent.TimeUnit unit)
                                     throws java.lang.InterruptedException,
                                            java.util.concurrent.TimeoutException
        Throws:
        java.lang.InterruptedException
        java.util.concurrent.TimeoutException

      • reload

        public void reload()

      • getBackgroundCompactionTaskSubmitter

        public static java.lang.Runnable getBackgroundCompactionTaskSubmitter()

      • setCompactionParameters

        public void setCompactionParameters​(java.util.Map<java.lang.String,​java.lang.String> options)
        Description copied from interface: ColumnFamilyStoreMBean
        Sets the compaction parameters locally for this node Note that this will be set until an ALTER with compaction = {..} is executed or the node is restarted
        Specified by:
        setCompactionParameters in interface ColumnFamilyStoreMBean
        Parameters:
        options - compaction options map

      • setCompactionParametersJson

        public void setCompactionParametersJson​(java.lang.String options)
        Description copied from interface: ColumnFamilyStoreMBean
        Sets the compaction parameters locally for this node Note that this will be set until an ALTER with compaction = {..} is executed or the node is restarted
        Specified by:
        setCompactionParametersJson in interface ColumnFamilyStoreMBean
        Parameters:
        options - compaction options with the same syntax as when doing ALTER ... WITH compaction = {..}

      • setCompressionParameters

        public void setCompressionParameters​(java.util.Map<java.lang.String,​java.lang.String> opts)
        Description copied from interface: ColumnFamilyStoreMBean
        Set the compression parameters locally for this node
        Specified by:
        setCompressionParameters in interface ColumnFamilyStoreMBean
        Parameters:
        opts - map of string names to values

      • getTableMBeanName

        public static java.lang.String getTableMBeanName​(java.lang.String ks,
                                                 java.lang.String name,
                                                 boolean isIndex)

      • getColumnFamilieMBeanName

        public static java.lang.String getColumnFamilieMBeanName​(java.lang.String ks,
                                                         java.lang.String name,
                                                         boolean isIndex)

      • updateSpeculationThreshold

        public void updateSpeculationThreshold()

      • getDataPaths

        public java.util.List<java.lang.String> getDataPaths()
                                              throws java.io.IOException
        Specified by:
        getDataPaths in interface ColumnFamilyStoreMBean
        Throws:
        java.io.IOException

      • supportsEarlyOpen

        public boolean supportsEarlyOpen()

      • invalidate

        public void invalidate()
        call when dropping or renaming a CF. Performs mbean housekeeping and invalidates CFS to other operations

      • invalidate

        public void invalidate​(boolean expectMBean)

      • createColumnFamilyStore

        public static ColumnFamilyStore createColumnFamilyStore​(Keyspace keyspace,
                                                        java.lang.String columnFamily,
                                                        TableMetadataRef metadata,
                                                        Directories directories,
                                                        boolean loadSSTables,
                                                        boolean registerBookkeeping,
                                                        boolean offline)
        This is only directly used by offline tools

      • scrubDataDirectories

        public static void scrubDataDirectories​(TableMetadata metadata)
                                 throws StartupException
        Removes unnecessary files from the cf directory at startup: these include temp files, orphans, zero-length files and compacted sstables. Files that cannot be recognized will be ignored.
        Throws:
        StartupException

      • loadNewSSTables

        public static void loadNewSSTables​(java.lang.String ksName,
                                   java.lang.String cfName)
        See #StorageService.importNewSSTables for more info
        Parameters:
        ksName - The keyspace name
        cfName - The columnFamily name

      • importNewSSTables

        public java.util.List<java.lang.String> importNewSSTables​(java.util.Set<java.lang.String> srcPaths,
                                                          boolean resetLevel,
                                                          boolean clearRepaired,
                                                          boolean verifySSTables,
                                                          boolean verifyTokens,
                                                          boolean invalidateCaches,
                                                          boolean extendedVerify,
                                                          boolean copyData)
        #Load new sstables from the given directory
        Specified by:
        importNewSSTables in interface ColumnFamilyStoreMBean
        Parameters:
        srcPaths - the path to the new sstables - if it is an empty set, the data directories will be scanned
        resetLevel - if the level should be reset to 0 on the new sstables
        clearRepaired - if repaired info should be wiped from the new sstables
        verifySSTables - if the new sstables should be verified that they are not corrupt
        verifyTokens - if the tokens in the new sstables should be verified that they are owned by the current node
        invalidateCaches - if row cache should be invalidated for the keys in the new sstables
        extendedVerify - if we should run an extended verify checking all values in the new sstables
        copyData - if we should copy data from source paths instead of moving them
        Returns:
        list of failed import directories

      • importNewSSTables

        public java.util.List<java.lang.String> importNewSSTables​(java.util.Set<java.lang.String> srcPaths,
                                                          boolean resetLevel,
                                                          boolean clearRepaired,
                                                          boolean verifySSTables,
                                                          boolean verifyTokens,
                                                          boolean invalidateCaches,
                                                          boolean extendedVerify)
        Description copied from interface: ColumnFamilyStoreMBean
        Load new sstables from the given directory
        Specified by:
        importNewSSTables in interface ColumnFamilyStoreMBean
        Parameters:
        srcPaths - the path to the new sstables - if it is an empty set, the data directories will be scanned
        resetLevel - if the level should be reset to 0 on the new sstables
        clearRepaired - if repaired info should be wiped from the new sstables
        verifySSTables - if the new sstables should be verified that they are not corrupt
        verifyTokens - if the tokens in the new sstables should be verified that they are owned by the current node
        invalidateCaches - if row cache should be invalidated for the keys in the new sstables
        extendedVerify - if we should run an extended verify checking all values in the new sstables
        Returns:
        list of failed import directories

      • rebuildSecondaryIndex

        public void rebuildSecondaryIndex​(java.lang.String idxName)

      • rebuildSecondaryIndex

        public static void rebuildSecondaryIndex​(java.lang.String ksName,
                                         java.lang.String cfName,
                                         java.lang.String... idxNames)

      • newSSTableDescriptor

        public Descriptor newSSTableDescriptor​(java.io.File directory)

      • switchMemtableIfCurrent

        public com.google.common.util.concurrent.ListenableFuture<CommitLogPosition> switchMemtableIfCurrent​(Memtable memtable)
        Switches the memtable iff the live memtable is the one provided
        Parameters:
        memtable -

      • switchMemtable

        public com.google.common.util.concurrent.ListenableFuture<CommitLogPosition> switchMemtable()

      • forceFlush

        public com.google.common.util.concurrent.ListenableFuture<CommitLogPosition> forceFlush()
        Flush if there is unflushed data in the memtables
        Returns:
        a Future yielding the commit log position that can be guaranteed to have been successfully written to sstables for this table once the future completes

      • forceFlush

        public com.google.common.util.concurrent.ListenableFuture<?> forceFlush​(CommitLogPosition flushIfDirtyBefore)
        Flush if there is unflushed data that was written to the CommitLog before @param flushIfDirtyBefore (inclusive).
        Returns:
        a Future yielding the commit log position that can be guaranteed to have been successfully written to sstables for this table once the future completes

      • flushLargestMemtable

        public static java.util.concurrent.CompletableFuture<java.lang.Boolean> flushLargestMemtable()
        Finds the largest memtable, as a percentage of *either* on- or off-heap memory limits, and immediately queues it for flushing. If the memtable selected is flushed before this completes, no work is done.

      • apply

        public void apply​(PartitionUpdate update,
                  UpdateTransaction indexer,
                  OpOrder.Group opGroup,
                  CommitLogPosition commitLogPosition)
        Insert/Update the column family for this key. Caller is responsible for acquiring Keyspace.switchLock param @ lock - lock that needs to be used. param @ key - key for update/insert param @ columnFamily - columnFamily changes

      • getOverlappingLiveSSTables

        public java.util.Collection<SSTableReader> getOverlappingLiveSSTables​(java.lang.Iterable<SSTableReader> sstables)
        Parameters:
        sstables -
        Returns:
        sstables whose key range overlaps with that of the given sstables, not including itself. (The given sstables may or may not overlap with each other.)

      • getAndReferenceOverlappingLiveSSTables

        public Refs<SSTableReader> getAndReferenceOverlappingLiveSSTables​(java.lang.Iterable<SSTableReader> sstables)
        like getOverlappingSSTables, but acquires references before returning

      • addSSTables

        public void addSSTables​(java.util.Collection<SSTableReader> sstables)

      • getExpectedCompactedFileSize

        public long getExpectedCompactedFileSize​(java.lang.Iterable<SSTableReader> sstables,
                                         OperationType operation)
        Calculate expected file size of SSTable after compaction. If operation type is CLEANUP and we're not dealing with an index sstable, then we calculate expected file size with checking token range to be eliminated. Otherwise, we just add up all the files' size, which is the worst case file size for compaction of all the list of files given.
        Parameters:
        sstables - SSTables to calculate expected compacted file size
        operation - Operation type
        Returns:
        Expected file size of SSTable after compaction

      • forceCleanup

        public CompactionManager.AllSSTableOpStatus forceCleanup​(int jobs)
                                                  throws java.util.concurrent.ExecutionException,
                                                         java.lang.InterruptedException
        Throws:
        java.util.concurrent.ExecutionException
        java.lang.InterruptedException

      • scrub

        public CompactionManager.AllSSTableOpStatus scrub​(boolean disableSnapshot,
                                                  boolean skipCorrupted,
                                                  boolean checkData,
                                                  boolean reinsertOverflowedTTL,
                                                  int jobs)
                                           throws java.util.concurrent.ExecutionException,
                                                  java.lang.InterruptedException
        Throws:
        java.util.concurrent.ExecutionException
        java.lang.InterruptedException

      • scrub

        public CompactionManager.AllSSTableOpStatus scrub​(boolean disableSnapshot,
                                                  boolean skipCorrupted,
                                                  boolean reinsertOverflowedTTL,
                                                  boolean alwaysFail,
                                                  boolean checkData,
                                                  int jobs)
                                           throws java.util.concurrent.ExecutionException,
                                                  java.lang.InterruptedException
        Throws:
        java.util.concurrent.ExecutionException
        java.lang.InterruptedException

      • rebuildOnFailedScrub

        public boolean rebuildOnFailedScrub​(java.lang.Throwable failure)
        CASSANDRA-5174 : For an index cfs we may be able to discard everything and just rebuild the index when a scrub fails.
        Returns:
        true if we are an index cfs and we successfully rebuilt the index

      • sstablesRewrite

        public CompactionManager.AllSSTableOpStatus sstablesRewrite​(boolean excludeCurrentVersion,
                                                            int jobs)
                                                     throws java.util.concurrent.ExecutionException,
                                                            java.lang.InterruptedException
        Throws:
        java.util.concurrent.ExecutionException
        java.lang.InterruptedException

      • relocateSSTables

        public CompactionManager.AllSSTableOpStatus relocateSSTables​(int jobs)
                                                      throws java.util.concurrent.ExecutionException,
                                                             java.lang.InterruptedException
        Throws:
        java.util.concurrent.ExecutionException
        java.lang.InterruptedException

      • isValid

        public boolean isValid()

      • getTracker

        public Tracker getTracker()
        Package protected for access from the CompactionManager.

      • getLiveSSTables

        public java.util.Set<SSTableReader> getLiveSSTables()

      • getUncompactingSSTables

        public java.lang.Iterable<SSTableReader> getUncompactingSSTables()

      • getPendingRepairStats

        public java.util.Map<java.util.UUID,​PendingStat> getPendingRepairStats()

      • releaseRepairData

        public CleanupSummary releaseRepairData​(java.util.Collection<java.util.UUID> sessions,
                                        boolean force)
        promotes (or demotes) data attached to an incremental repair session that has either completed successfully, or failed
        Returns:
        session ids whose data could not be released

      • gcBefore

        public int gcBefore​(int nowInSec)

      • getSSTablesForKey

        public java.util.List<java.lang.String> getSSTablesForKey​(java.lang.String key)
        Description copied from interface: ColumnFamilyStoreMBean
        Returns a list of filenames that contain the given key on this node
        Specified by:
        getSSTablesForKey in interface ColumnFamilyStoreMBean
        Returns:
        list of filenames containing the key

      • getSSTablesForKey

        public java.util.List<java.lang.String> getSSTablesForKey​(java.lang.String key,
                                                          boolean hexFormat)
        Description copied from interface: ColumnFamilyStoreMBean
        Returns a list of filenames that contain the given key on this node
        Specified by:
        getSSTablesForKey in interface ColumnFamilyStoreMBean
        hexFormat - if key is in hex string format
        Returns:
        list of filenames containing the key

      • beginLocalSampling

        public void beginLocalSampling​(java.lang.String sampler,
                               int capacity,
                               int durationMillis)
        Description copied from interface: ColumnFamilyStoreMBean
        begin sampling for a specific sampler with a given capacity. The cardinality may be larger than the capacity, but depending on the use case it may affect its accuracy
        Specified by:
        beginLocalSampling in interface ColumnFamilyStoreMBean

      • finishLocalSampling

        public java.util.List<javax.management.openmbean.CompositeData> finishLocalSampling​(java.lang.String sampler,
                                                                                    int count)
                                                                             throws javax.management.openmbean.OpenDataException
        Specified by:
        finishLocalSampling in interface ColumnFamilyStoreMBean
        Returns:
        top count items for the sampler since beginLocalSampling was called
        Throws:
        javax.management.openmbean.OpenDataException

      • cleanupCache

        public void cleanupCache()

      • snapshotWithoutFlush

        public void snapshotWithoutFlush​(java.lang.String snapshotName)

      • snapshotWithoutFlush

        public java.util.Set<SSTableReader> snapshotWithoutFlush​(java.lang.String snapshotName,
                                                         com.google.common.base.Predicate<SSTableReader> predicate,
                                                         boolean ephemeral,
                                                         com.google.common.util.concurrent.RateLimiter rateLimiter)
        Parameters:
        ephemeral - If this flag is set to true, the snapshot will be cleaned during next startup

      • clearEphemeralSnapshots

        protected static void clearEphemeralSnapshots​(Directories directories)

      • getSnapshotSSTableReaders

        public Refs<SSTableReader> getSnapshotSSTableReaders​(java.lang.String tag)
                                              throws java.io.IOException
        Throws:
        java.io.IOException

      • snapshot

        public java.util.Set<SSTableReader> snapshot​(java.lang.String snapshotName)
        Take a snap shot of this columnfamily store.
        Parameters:
        snapshotName - the name of the associated with the snapshot

      • snapshot

        public java.util.Set<SSTableReader> snapshot​(java.lang.String snapshotName,
                                             boolean skipFlush,
                                             com.google.common.util.concurrent.RateLimiter rateLimiter)
        Take a snap shot of this columnfamily store.
        Parameters:
        snapshotName - the name of the associated with the snapshot
        skipFlush - Skip blocking flush of memtable
        rateLimiter - Rate limiter for hardlinks-per-second

      • snapshot

        public java.util.Set<SSTableReader> snapshot​(java.lang.String snapshotName,
                                             com.google.common.base.Predicate<SSTableReader> predicate,
                                             boolean ephemeral,
                                             boolean skipFlush)
        Parameters:
        ephemeral - If this flag is set to true, the snapshot will be cleaned up during next startup
        skipFlush - Skip blocking flush of memtable

      • snapshot

        public java.util.Set<SSTableReader> snapshot​(java.lang.String snapshotName,
                                             com.google.common.base.Predicate<SSTableReader> predicate,
                                             boolean ephemeral,
                                             boolean skipFlush,
                                             com.google.common.util.concurrent.RateLimiter rateLimiter)
        Parameters:
        ephemeral - If this flag is set to true, the snapshot will be cleaned up during next startup
        skipFlush - Skip blocking flush of memtable
        rateLimiter - Rate limiter for hardlinks-per-second

      • snapshotExists

        public boolean snapshotExists​(java.lang.String snapshotName)

      • getSnapshotCreationTime

        public long getSnapshotCreationTime​(java.lang.String snapshotName)

      • clearSnapshot

        public void clearSnapshot​(java.lang.String snapshotName)
        Clear all the snapshots for a given column family.
        Parameters:
        snapshotName - the user supplied snapshot name. If left empty, all the snapshots will be cleaned.

      • getSnapshotDetails

        public java.util.Map<java.lang.String,​Directories.SnapshotSizeDetails> getSnapshotDetails()
        Returns:
        Return a map of all snapshots to space being used The pair for a snapshot has true size and size on disk.

      • getRawCachedPartition

        public CachedPartition getRawCachedPartition​(DecoratedKey key)
        Returns:
        the cached partition for @param key if it is already present in the cache. Not that this will not readAndCache the parition if it is not present, nor are these calls counted in cache statistics. Note that this WILL cause deserialization of a SerializingCache partition, so if all you need to know is whether a partition is present or not, use containsCachedParition instead.

      • invalidateRowCache

        public int invalidateRowCache​(java.util.Collection<Bounds<Token>> boundsToInvalidate)

      • invalidateCounterCache

        public int invalidateCounterCache​(java.util.Collection<Bounds<Token>> boundsToInvalidate)

      • containsCachedParition

        public boolean containsCachedParition​(DecoratedKey key)
        Returns:
        true if @param key is contained in the row cache

      • invalidateCachedPartition

        public void invalidateCachedPartition​(RowCacheKey key)

      • invalidateCachedPartition

        public void invalidateCachedPartition​(DecoratedKey key)

      • forceMajorCompaction

        public void forceMajorCompaction()

      • forceMajorCompaction

        public void forceMajorCompaction​(boolean splitOutput)
        Description copied from interface: ColumnFamilyStoreMBean
        force a major compaction of this column family
        Specified by:
        forceMajorCompaction in interface ColumnFamilyStoreMBean
        Parameters:
        splitOutput - true if the output of the major compaction should be split in several sstables

      • forceCompactionForTokenRange

        public void forceCompactionForTokenRange​(java.util.Collection<Range<Token>> tokenRanges)
                                  throws java.util.concurrent.ExecutionException,
                                         java.lang.InterruptedException
        Description copied from interface: ColumnFamilyStoreMBean
        force a major compaction of specified key range in this column family
        Specified by:
        forceCompactionForTokenRange in interface ColumnFamilyStoreMBean
        Throws:
        java.util.concurrent.ExecutionException
        java.lang.InterruptedException

      • estimatedKeysForRange

        public long estimatedKeysForRange​(Range<Token> range)

      • clearUnsafe

        public void clearUnsafe()
        For testing. No effort is made to clear historical or even the current memtables, nor for thread safety. All we do is wipe the sstable containers clean, while leaving the actual data files present on disk. (This allows tests to easily call loadNewSSTables on them.)

      • truncateBlocking

        public void truncateBlocking()

      • truncateBlockingWithoutSnapshot

        public void truncateBlockingWithoutSnapshot()

      • dumpMemtable

        public java.util.concurrent.Future<CommitLogPosition> dumpMemtable()
        Drops current memtable without flushing to disk. This should only be called when truncating a column family which is not durable.

      • runWithCompactionsDisabled

        public <V> V runWithCompactionsDisabled​(java.util.concurrent.Callable<V> callable,
                                        boolean interruptValidation,
                                        boolean interruptViews)

      • runWithCompactionsDisabled

        public <V> V runWithCompactionsDisabled​(java.util.concurrent.Callable<V> callable,
                                        com.google.common.base.Predicate<SSTableReader> sstablesPredicate,
                                        boolean interruptValidation,
                                        boolean interruptViews,
                                        boolean interruptIndexes)
        Runs callable with compactions paused and compactions including sstables matching sstablePredicate stopped
        Parameters:
        callable - what to do when compactions are paused
        sstablesPredicate - which sstables should we cancel compactions for
        interruptValidation - if we should interrupt validation compactions
        interruptViews - if we should interrupt view compactions
        interruptIndexes - if we should interrupt compactions on indexes. NOTE: if you set this to true your sstablePredicate must be able to handle LocalPartitioner sstables!

      • toString

        public java.lang.String toString()
        Overrides:
        toString in class java.lang.Object

      • disableAutoCompaction

        public void disableAutoCompaction()

      • enableAutoCompaction

        public void enableAutoCompaction()

      • enableAutoCompaction

        public void enableAutoCompaction​(boolean waitForFutures)
        used for tests - to be able to check things after a minor compaction
        Parameters:
        waitForFutures - if we should block until autocompaction is done

      • getCrcCheckChance

        public java.lang.Double getCrcCheckChance()

      • getMeanEstimatedCellPerPartitionCount

        public int getMeanEstimatedCellPerPartitionCount()

      • getMeanPartitionSize

        public double getMeanPartitionSize()

      • getMeanRowCount

        public int getMeanRowCount()

      • decorateKey

        public DecoratedKey decorateKey​(java.nio.ByteBuffer key)

      • isIndex

        public boolean isIndex()
        true if this CFS contains secondary index data

      • concatWithIndexes

        public java.lang.Iterable<ColumnFamilyStore> concatWithIndexes()

      • getBuiltIndexes

        public java.util.List<java.lang.String> getBuiltIndexes()
        Description copied from interface: ColumnFamilyStoreMBean
        Returns a list of the names of the built column indexes for current store
        Specified by:
        getBuiltIndexes in interface ColumnFamilyStoreMBean
        Returns:
        list of the index names

      • getUnleveledSSTables

        public int getUnleveledSSTables()
        Specified by:
        getUnleveledSSTables in interface ColumnFamilyStoreMBean
        Returns:
        the number of SSTables in L0. Always return 0 if Leveled compaction is not enabled.

      • getSSTableCountPerLevel

        public int[] getSSTableCountPerLevel()
        Specified by:
        getSSTableCountPerLevel in interface ColumnFamilyStoreMBean
        Returns:
        sstable count for each level. null unless leveled compaction is used. array index corresponds to level(int[0] is for level 0, ...).

      • isEmpty

        public boolean isEmpty()

      • isRowCacheEnabled

        public boolean isRowCacheEnabled()

      • isCounterCacheEnabled

        public boolean isCounterCacheEnabled()

      • isKeyCacheEnabled

        public boolean isKeyCacheEnabled()

      • discardSSTables

        public void discardSSTables​(long truncatedAt)
        Discard all SSTables that were created before given timestamp. Caller should first ensure that comapctions have quiesced.
        Parameters:
        truncatedAt - The timestamp of the truncation (all SSTables before that timestamp are going be marked as compacted)

      • trueSnapshotsSize

        public long trueSnapshotsSize()
        Specified by:
        trueSnapshotsSize in interface ColumnFamilyStoreMBean
        Returns:
        the size of SSTables in "snapshots" subdirectory which aren't live anymore

      • getIfExists

        public static ColumnFamilyStore getIfExists​(TableId id)
        Returns a ColumnFamilyStore by id if it exists, null otherwise Differently from others, this method does not throw exception if the table does not exist.

      • getIfExists

        public static ColumnFamilyStore getIfExists​(java.lang.String ksName,
                                            java.lang.String cfName)
        Returns a ColumnFamilyStore by ksname and cfname if it exists, null otherwise Differently from others, this method does not throw exception if the keyspace or table does not exist.

      • invalidateDiskBoundaries

        public void invalidateDiskBoundaries()

      • hasMisplacedSSTables

        public boolean hasMisplacedSSTables()
        Description copied from interface: ColumnFamilyStoreMBean
        Check SSTables whether or not they are misplaced.
        Specified by:
        hasMisplacedSSTables in interface ColumnFamilyStoreMBean
        Returns:
        true if any of the SSTables is misplaced. If all SSTables are correctly placed or the partitioner does not support splitting, it returns false.