com.google.enterprise.connector.util.diffing

Class DocumentSnapshotRepositoryMonitor

java.lang.Object

com.google.enterprise.connector.util.diffing.DocumentSnapshotRepositoryMonitor

All Implemented Interfaces:

java.lang.Runnable

public class DocumentSnapshotRepositoryMonitor
extends java.lang.Object
implements java.lang.Runnable

A service that monitors a SnapshotRepository and makes callbacks when changes occur.

This implementation works as follows. It repeatedly scans all the DocumentSnapshot entries returned by SnapshotRepository.iterator(). On each pass, it compares the current contents of the repository to a record of what it saw on the previous pass. The record is stored as a file in the local repository. Each discrepancy is propagated to the client.

Using a local snapshot of the repository has some serious flaws for continuous crawl:

• The local snapshot can diverge from the actual contents of the GSA. This can lead to situations where discrepancies are not corrected.
• If the local snapshot gets corrupted, there is no way to recover short of deleting all on the GSA and starting again.

A much more robust solution is to obtain snapshots directly from the GSA at least part of the time. (However, to save bandwidth, it may still be useful to keep local snapshots and only get an "authoritative" snapshot from the cloud occasionally. E.g., once a week or if the local snapshot is corrupted.)

When an API to do that is available, this implementation should be fixed to use it.

Since:

2.8

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static interface	DocumentSnapshotRepositoryMonitor.Callback The client provides an implementation of this interface to receive notification of changes to the repository.

Constructor Summary

Constructors

Constructor and Description
DocumentSnapshotRepositoryMonitor(java.lang.String name, SnapshotRepository<? extends DocumentSnapshot> query, SnapshotStore snapshotStore, DocumentSnapshotRepositoryMonitor.Callback callback, DocumentSink documentSink, MonitorCheckpoint initialCp, DocumentSnapshotFactory documentSnapshotFactory) Creates a DocumentSnapshotRepositoryMonitor that monitors the Repository rooted at root.

Method Summary

Modifier and Type	Method and Description
void	acceptGuarantee(MonitorCheckpoint cp)
void	run()
void	setTraversalSchedule(TraversalSchedule traversalSchedule)
void	shutdown()
void	testTraversalSchedule()

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DocumentSnapshotRepositoryMonitor

public DocumentSnapshotRepositoryMonitor(java.lang.String name,
                                         SnapshotRepository<? extends DocumentSnapshot> query,
                                         SnapshotStore snapshotStore,
                                         DocumentSnapshotRepositoryMonitor.Callback callback,
                                         DocumentSink documentSink,
                                         MonitorCheckpoint initialCp,
                                         DocumentSnapshotFactory documentSnapshotFactory)

Creates a DocumentSnapshotRepositoryMonitor that monitors the Repository rooted at root.

Parameters:

name - the name of this monitor (a hash of the start path)

query - query for files

snapshotStore - where snapshots are stored

callback - client callback

documentSink - destination for filtered out file info

initialCp - checkpoint when system initiated, could be null

documentSnapshotFactory - for un-serializing DocumentSnapshot objects.

Method Detail

run

public void run()

Specified by:

run in interface java.lang.Runnable

acceptGuarantee

public void acceptGuarantee(MonitorCheckpoint cp)

testTraversalSchedule

public void testTraversalSchedule()
                           throws java.lang.NullPointerException,
                                  java.lang.InterruptedException

Throws:

java.lang.NullPointerException

java.lang.InterruptedException

shutdown

public void shutdown()

setTraversalSchedule

public void setTraversalSchedule(TraversalSchedule traversalSchedule)