Online Store Automation - Populating Our Store (Part 3)

To keep populating my store, I need a way to get lists of new titles (or previously unlisted titles) to add to my database. For this, I take advantage of the Amazon Web Services (or AWS) and the fact that for each title in their list, they keep a list of similar titles. So, as I update the stock/pricing information for each title, I can query Amazon to see if there are any similar titles to this one that I had not already covered. Here is the code that accomplishes this.

   1:                          // Get similar titles
   2:                          AmazonNodeLister lister = new AmazonNodeLister();
   3:                          List<string> similar = lister.GetSimilarProducts(curAmazonItem.Asin);
   4:                          foreach (string simitem in similar)
   5:                          {
   6:                              if ((simitem.Length == 10) && (System.Char.IsDigit(simitem[0])) && (!File.Exists(textEditData.Text + "\\" + simitem + ".dat")))
   7:                              {
   8:                                  AddStatus(" Similar = " + simitem);
   9:   
  10:                                  AmazonItem amazonItem = new AmazonItem(simitem);
  11:                                  file = File.Open(textEditData.Text + "\\" + simitem + ".dat", FileMode.Create);
  12:                                  formatter = (IFormatter)new BinaryFormatter();
  13:                                  formatter.Serialize(file, curAmazonItem);
  14:                                  file.Close();
  15:   
  16:                                  WorkItem workItem = new WorkItem();
  17:                                  workItem.Type = WorkItem.WorkType.QueryIngram;
  18:                                  workItem.Data = textEditData.Text + "\\" + simitem + ".dat";
  19:                                  workList.Enqueue(workItem);
  20:                              }
  21:                          }
  22:   
  23:   
  24:          public List<string> GetSimilarProducts(string asin)
  25:          {
  26:              SimilarityLookup lookup = new SimilarityLookup();
  27:              lookup.AssociateTag = "FOO";
  28:              lookup.AWSAccessKeyId = "BAR";
  29:              lookup.Shared = new SimilarityLookupRequest();
  30:              lookup.Shared.ItemId = new string[2];
  31:              lookup.Shared.ItemId[0] = asin;
  32:              lookup.Shared.ItemId[1] = null;
  33:              lookup.Shared.ResponseGroup = new string[3];
  34:              lookup.Shared.ResponseGroup[0] = "Request";
  35:              lookup.Shared.ResponseGroup[1] = "ItemAttributes";
  36:              lookup.Shared.ResponseGroup[2] = null;
  37:              SimilarityLookupResponse resp = service.SimilarityLookup(lookup);
  38:   
  39:              List<string> res = new List<string>();
  40:              if ((resp.Items != null) && (resp.Items.Length >= 1) && (resp.Items[0].Item != null))
  41:              {
  42:                  foreach (Item item in resp.Items[0].Item)
  43:                  {
  44:                      if ( (item.ItemAttributes != null) && (item.ItemAttributes.ISBN != null))
  45:                          res.Add(item.ASIN);
  46:                  }
  47:              }
  48:   
  49:              System.Threading.Thread.Sleep(500);
  50:              return res;
  51:          }
  52:   

This code essentially asks Amazon for product info on a specific ASIN (which is the Amazon SKU). In the case of a book, the ASIN is essentially the same as the ISBN number for the book. As part of the product information, I can request a list of similar products as suggested by Amazon. In this case, this is simplified by asking amazon for the list of Similar products through a "SimilarityLookup". Since similar products aren't necessarily books, I verify that the returned products do have an ISBN number assigned to them. Keep in mind that ISBN numbers also include things such as CDs and DVDs but I filter those out at some other point.

Oh and a final note on why I call sleep on the current thread at the end. Amazon restricts queries roughly to one per second from a specific IP address. I sleep for 1/2 second as I do other tasks between queries, this allows me to throttle my traffic to the AWS.

For more info on the Amazon Web Services, check out the following: http://aws.amazon.com/