Thursday, January 9, 2014

Learning SolrCloud Code

org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(SolrQueryRequest, SolrQueryResponse)
    UpdateRequestProcessor processor = processorChain.createProcessor(req, rsp);
org.apache.solr.update.processor.UpdateRequestProcessorChain.createProcessor(SolrQueryRequest, SolrQueryResponse)

LogUpdateProcessorFactory
==> DistributedUpdateProcessorFactory will be inserted before RunUpdateProcessorFactory
RunUpdateProcessorFactory

RunUpdateProcessorFactory
if (zkEnabled) {
cmdDistrib = new SolrCmdDistributor(coreDesc.getCoreContainer().getUpdateExecutor());
}

org.apache.solr.update.processor.DistributedUpdateProcessor.setupRequest(String, SolrInputDocument)
      ClusterState cstate = zkController.getClusterState();
      DocCollection coll = cstate.getCollection(collection);
      Slice slice = coll.getRouter().getTargetSlice(id, doc, req.getParams(), coll);


org.apache.solr.common.cloud.HashBasedRouter.getTargetSlice(String, SolrInputDocument, SolrParams, DocCollection)
public Slice getTargetSlice(String id, SolrInputDocument sdoc, SolrParams params, DocCollection collection) {
if (id == null) id = getId(sdoc, params);
int hash = sliceHash(id, sdoc, params,collection);
return hashToSlice(hash, collection);
}

protected Slice hashToSlice(int hash, DocCollection collection) {
for (Slice slice : collection.getActiveSlices()) {
Range range = slice.getRange();
if (range != null && range.includes(hash)) return slice;
}
}

org.apache.solr.common.cloud.DocRouter.getTargetSlice(String, SolrInputDocument, SolrParams, DocCollection)


org.apache.solr.update.AddUpdateCommand.getHashableId()

org.apache.solr.update.processor.DistributedUpdateProcessor.setupRequest(String, SolrInputDocument)

org.apache.solr.update.SolrCmdDistributor.distribAdd(AddUpdateCommand, List<Node>, ModifiableSolrParams, boolean)
  public void distribAdd(AddUpdateCommand cmd, List<Node> nodes, ModifiableSolrParams params, boolean synchronous) throws IOException {

    for (Node node : nodes) {
      UpdateRequest uReq = new UpdateRequest();
      uReq.setParams(params);
      uReq.add(cmd.solrDoc, cmd.commitWithin, cmd.overwrite);
      submit(new Req(node, uReq, synchronous));
    }
    
  }

http://stackoverflow.com/questions/13433420/solrcloud-expanding-index-without-downtime
Under the assumption that expanding index means adding new shards, then the answer is that it is not possible to create new shards without a reindexing. This is because sharding is done using hashing and adding another bucket (shard) requires the hash function to be changed in order for new items into the shard.

<!> Solr4.3 The new "SPLITSHARD" collection API can be used to split an existing shard into two shards containing exactly half the range of the parent shard each. More details can be found under the "Managing collections via the Collections API" section.

http://stackoverflow.com/questions/14784326/solr-4-adding-shard-to-existing-cluster

Custom Hashing
https://issues.apache.org/jira/browse/SOLR-2592

http://java.dzone.com/articles/solrcloud-what-happens-when

http://java.dzone.com/articles/using-solrj-basic

No comments:

Post a Comment