This is just a quick update to let you know that the background checks that power the hourly data collection for NovelRank have been completely rewritten. While they have had improvements and tweaks over the last 6 years, this new design is leaps and bounds better since being rewritten.
Benefits:
- Each domain runs its own independent script. Previously, only Amazon.com and Amazon.de were independent. This means no more books on say, Amazon.fr, that are active but don’t have a new sales rank for days or weeks due to script inefficiency.
- More books checked per hour. When NovelRank started, Amazon restricted API requests to a certain number per hour. Thus, NovelRank’s scripts were designed around that. Now, they are throttled based on API calls/second. Thus, the new scripts can slow down when Amazon asks, but otherwise make better use of every second in every hour to get sales rank data.
- PRO user books every hour, on the hour. Previously, NovelRank put PRO books first as promised, however it would then run the rest of the books which sometimes took more than 1 hour to complete. It would not check PRO books again until that was done, so PRO books would be first, but still may miss an hour of sales rank data. Now, as soon as it is a new hour, PRO books are processed, pausing any current processing, ensuring hourly sales rank data at a consistent time of the hour, each hour.
Throttling
Much like before, to conserve API requests, NovelRank throttles how often it checks sales rank for a book based on the last time it had an estimated sale on that domain. Here’s the breakdown:
- Sale in the last 3 days: Every loop (approximately hourly)
- Sale in-between 3 days and 10 days: Every 3rd loop (approximately every 3 hours)
- Sale in-between 10 days and 30 days: Every 6th loop (approximately every 6 hours)
- Sale more than 30 days ago: Every 12th loop (approximately every 12 hours)
Please note that for Amazon.com, due the exceptionally large number of books, the approximate times will be hourly, every 6 hours, every 12 hours, every 24 hours, respectively.
Final Summary
This should prove to be a marked improvement across the board for customers, especially for non-US, and non-DE customers. It should also improve the offering for the paying PRO subscribers. Enjoy. :)
