Easy Material Website Change Monitoring

Goal: Sometimes you need to be alerted when a website string is present or absent such as “in stock” or “available”. Here I outline a quick and easy method to alert you to such changes specifically in JSON data using just a free Chrome browser extension.

Instead of paying a monthly fee for a powerful change detection service, here I outline a quick and easy method to alert you to the presence or absence of a regex string specifically in JSON data using just a free Chrome browser extension. As a bonus, you don’t need to set (spoof) headers or set cookies, and you can even be logged in to a site. You will likely avoid reCAPTCHA as well. Plus you likely won’t tread over the TOS as you aren’t using a third-party service.

I’ll go through two case studies where I actually use this technique:

  1. BestBuy Item In-Stock Notification – Know when an item is in-stock
  2. Expired Domain Availability Notification – Know when an expired domain is available

You’ll need:

With Auto Refresh Plus you can set an interval to reload the current web page tab (thus your headers and cookies are persisted) and be alerted to a regex string that is either absent or present. This alert can be music that plays continuously until you stop it. JSON Viewer just makes JSON pretty to look at so it is optional.

Case Study 1: BestBuy Item In-Stock Notification

BestBuy had a Boxing Week sale on some memory cards I had my eye on. The problem is that the stores nearest to me were out of stock and often stayed out of stock. I suspect this was by design because Amazon also had sales on the the same cards and BestBuy will price match (there is no price-match for online orders). However, when an item comes in stock, only those with eagle eyes or luck will notice and reserve one for in-store pickup. Let’s automate some eagle eyes.

BestBuy home page with Network panel open
BestBuy home page with the Network Inspector panel open

Step 1. Find the JSONP feed

You should have a look at both the XHR and JS panel. Look at the preview for obvious URLs and investigate further. Sometimes JSON is consumed by the site in the same domain, and other times it is JSONP to get around CORS1. Here our target is JSONP. Notice that the item is out of stock.

Find the JSONP product availability URL
Find the JSONP product availability URL

Step 2. Isolate the JSONP in a new tab

The JSONP here contains the location and the inventory status. In this case Richmond and OutOfStock. I use the JSON Viewer extension to get a feel for the formatted JSON structure so I can visually look for clues to lock on to. For example, what would the “In Stock” value look like? By looking around I know it is InStock.

Find the target structure with JSON Viewer
Find the target structure with JSON Viewer

When you have a rough idea what the target string should look like, switch to raw mode and isolate the exact string. Copy this to the clipboard for the next step.

Isolate the target string to find
Isolate the target string

Step 3. Set the Auto Refresh Plus settings once

This step only needs to be done once. Be sure to enable the Page Monitor and choose to alert you when it finds or can’t find a string/regex. I usually select Find. Then select a sound, make sure your speakers are on and test it, then set it to play until you dismiss the alert.

Set the Auto Page Refresh options once
Set the Auto Page Refresh options once

Step 4. Set the search regex string

Paste in the target string from step two, but change OutOfStock to InStock.

Set the interval and string to find
Set the interval and string to find

Since this is a regex field, be sure to escape special characters like brackets. Select a kind refresh rate, and click Start. You can experiment with regex strings, but this is sufficient for this case. Notice that an interval timer is initiated. The page will refresh when this timeout reaches zero and the timer will start again. This will continue until the target string is found.

Escape the string to a proper regex string
Escape the string to a proper regex string

Step 5. Wait for the alert

When the item is in stock, you should be treated to an alert like the one below. You can also test the alert with variants of the regex string to confirm they act as expected.

Found string alert notification
Found string alert notification

Case Study 2: Expired Domain Availability Notification

I had my eye on a long-held domain that was in a redemption period. From experience I calculated when the redemption period ends (about 1.5 months from expiry) and the domain goes into deletion. When it is deleted the various registrars have no Whois records2. While the deletion event is propagating the domain is “available” but no one can register it just yet. It is here that luck (or pressing F5 repeatedly for days) would come into play. Let’s automate a watchdog.

Step 1. Find the JSON feed

I use Namecheap.com because they saved me a lot of money when a big-name domain registrar tried to gouge me on domain renewals. For that I continue to give Namecheap my business. In this example let’s pretend someexample.com is in deletion. I’ll head to namecheap.com, open the Network inspector panel and search for the JSON feed. In this case the XHR tab rewards us with the JSON URL we seek.

Namecheap.com home page and network inspector panel
Namecheap.com home page and network inspector panel

Step 2. Isolate the JSON in a new tab

You don’t need to use the JSON Viewer pretty view here to notice that the .com domain is missing from the JSON. In fact, when you look at the home page in step one you can see that .net, .ai, .to, and .travel are available, just as in the JSON data below. What we should search for is plain and simple: someexample.com (Actually, someexample\.com for regex).

JSON domain availability information
JSON domain availability information

Step 3. Set the search string

Simply search for someexample\.com. Set a kind refresh interval again, and click Start.

Search for the unavailable domain as a regex string
Search for the unavailable domain as a regex string

Step 4. Wait for the alert

When the domain is truly available for purchase, it will appear in the JSON content and an alert will appear with the music. You should test the regex with a string that is already present.

Found domain alert notification
Found domain alert notification

In fact, just last week one morning around 3AM I heard the alert and I picked up a seven-letter dot-com domain that I alluded to earlier.

Discussion

This is easy and fun, especially because it requires a little detective work and puzzle solving. Please be considerate of the target website with the refresh interval. Some sites use a timestamp as a querystring parameter, so YMMV3. Finally. this is likely within the TOS of most websites because this technique is performed in your home Chrome browser, not a third-party web service.

These were just a couple case studies, but this technique can be used on several websites for quick and easy alerts when a ticket is available, a novel is in stock, a flash sale is taking place, an IIROC4 stock trading halt is in effect, an RMT cancellation creates an open booking, and so on. Enjoy.

Notes:

  1. CORS = Cross-Origin Resource Sharing
  2. Whois records are domain ownership records
  3. Your Mileage May Vary
  4. Investment Industry Organization of Canada