Constructor Detail

• NamedResourceFirehose

  public NamedResourceFirehose()

Method Detail

• initConfig

  public void initConfig(Config config)

  Description copied from class: AbstractAdaptor
  Provides the opportunity for the Adaptor to create new configuration values or override default values. Only Config.addKey(java.lang.String, java.lang.String) should likely be called. The user's configuration will override any values set in this way. This method is called by AbstractAdaptor.main(com.google.enterprise.adaptor.Adaptor, java.lang.String[]) before Adaptor.init(com.google.enterprise.adaptor.AdaptorContext) is called.

  This implementation does nothing.

  Specified by:

  initConfig in interface Adaptor

  Overrides:

  initConfig in class AbstractAdaptor

• init

  public void init(AdaptorContext context)
            throws Exception

  Description copied from class: AbstractAdaptor
  Initialize adaptor with the current context. This is the ideal time to start any threads to do extra behind-the-scenes work. The context points to other useful objects that can be used at any time. For example, methods on DocIdPusher provided via AdaptorContext.getDocIdPusher() are allowed to be called whenever the Adaptor wishes. This allows doing event-based incremental pushes at any time.

  The method is called at the end of GsaCommunicationHandler.start(java.lang.Runnable).

  If you experience a fatal error during initialization, feel free to throw an Exception to cancel the startup process.

  This implementation does nothing.

  Specified by:

  init in interface Adaptor

  Overrides:

  init in class AbstractAdaptor

  Throws:

  Exception

• getDocIds

  public void getDocIds(DocIdPusher pusher)
                 throws InterruptedException

  Description copied from interface: Adaptor
  Pushes all the DocIds that are suppose to be indexed by the GSA. This will frequently involve re-sending DocIds to the GSA, but this allows healing previous errors and cache inconsistencies. Re-sending DocIds is very fast and should be considered free on the GSA. This method should determine a list of DocIds to push and call DocIdPusher.pushDocIds(java.lang.Iterable<com.google.enterprise.adaptor.DocId>) one or more times and DocIdPusher.pushNamedResources(java.util.Map<com.google.enterprise.adaptor.DocId, com.google.enterprise.adaptor.Acl>) if using named resources.

  pusher is provided as convenience and is the same object provided to Adaptor.init(com.google.enterprise.adaptor.AdaptorContext) previously. This method may take a while and implementations are free to call Thread.sleep(long) occasionally to reduce load.

  If you experience a fatal error, feel free to throw an IOException or RuntimeException. In the case of an error, the ExceptionHandler in use in AdaptorContext will determine if and when to retry.

  Specified by:

  getDocIds in interface Adaptor

  Throws:

  InterruptedException

• getDocContent

  public void getDocContent(Request req,
                   Response resp)
                     throws IOException

  Description copied from interface: Adaptor
  Provides contents and metadata of particular document. This method should be highly parallelizable and support twenty or more concurrent calls. Two to three concurrent calls may be average during initial GSA crawling, but twenty or more concurrent calls is typical when the GSA is recrawling unmodified content.

  If you experience a fatal error, feel free to throw an IOException or RuntimeException. In the case of an error, the GSA will determine if and when to retry.

  Specified by:

  getDocContent in interface Adaptor

  Throws:

  IOException

• getModifiedDocIds

  public void getModifiedDocIds(DocIdPusher pusher)
                         throws IOException,
                                InterruptedException

  Description copied from interface: PollingIncrementalLister
  Check for documents modified since the last call to the method. This method is intended to provide little-effort updates to the GSA about recent modifications. Providing updates here allows greatly decreasing the amount of latency before the GSA notices a document was added/modified/deleted, but nothing more. It does not need to be perfect since the GSA's recrawling will notice modifications and deletions and Adaptor.getDocIds(com.google.enterprise.adaptor.DocIdPusher) will provide additions.

  For the first invocation, implementations can simply provide a small amount of very recent history (e.g., last five revisions or last hour of modifications) or provide no history and just initialize data structures for future invocations. Adaptors are always encouraged to not persist state; providing recent history during the first invocation allows the adaptor to handle upgrades, computer restarts, and power outages without trouble and still provide low-latency for those time periods. If an hour is too short a period of time, then feel free to send a day's worth of history instead. However, remember that missing modifications here only increases the amount of latency before the GSA notices the modification.

  Specified by:

  getModifiedDocIds in interface PollingIncrementalLister

  Parameters:

  pusher - convenience reference to pusher

  Throws:

  IOException

  InterruptedException

• main

  public static void main(String[] args)

  Call default main for adaptors.