I was just testing a new release of twarc and was alerted to a test failure from Travis that seems to point to a change in Twitter's API. I thought I would quickly write it up because it seemed like an interesting bit of Twitter API arcana, as well as an unexpected use of black box testing to detect changes in social media platforms like Twitter. It also highlights (for me) the benefit of sometimes being lazy and not using mock objects in automated tests.

The specific test that is failing is test_follow which uses Twitter's follow streaming API request to watch for tweets from a handful of major news organizations like @guardian, @nytimes, @cnnbreak, etc. Once the test gets a tweet it examines the JSON and simply verifies that it came from one of the followed accounts. I added the test to make sure twarc was doing things correctly more than to test Twitter's API.

The weird thing is that the test has recently started to fail because sometimes it gets back tweets that don't appear to be sent from any of the followed accounts. The tweets don't even appear to be retweets or quotes of those accounts either. As you can see from the JSON included at the bottom of the test failure the tweet that is returned is a retweet from @Margaritin22 (971990761) of a tweet by @nytimesbusiness (1754641) but the test wasn't following either of those users...but it was following @nytimes (807095).

So it looks as if Twitter's follow API now has some (new?) logic that provides @nytimesbusiness tweets because I am following tweets from @nytimes. Perhaps this is some new marketing feature that allows media outlets like the New York Times to pay Twitter to bundle accounts together for promotional purposes? Or perhaps there is an algorithm at play that assumes that since I am following @nytimes I would be interested in @nytimesbusiness?

The test has only been in use since December 2015 which really isn't too long ago. Plus the tests normally only run when I'm actively developing twarc or when I push to GitHub. So perhaps this behavior isn't new and it has just been lucky that the test hasn't failed until now?

But this follow behavior feels a bit like some other changes that Twitter has made to the timeline where tweets are suggested based on popularity. If anyone has any insight into this please let me know.

For the moment the test has stopped following @nytimes and it appears to be working again...at least for now. Here's a musical interlude since you made this far: