Automating Blog Tasks – Bring All Your Skills to the Table
Every week, I publish my top five favorite posts from other bloggers. It’s a feature I call Other Posts to Crow About. My goal is to draw attention to posts my readers might like, but from blogs other than mine. I wrote a post that talks about how I format the posts, as well as the approach I take to marketing them on on social media. Now, I’d like to talk about how I get through all those sites. It has to do with automating blog tasks — in this case, going through my list of sites.
To write that post, I check all of the blogs in the Massive List of Anime Sites. Currently, it contains around 360 sites. I allocate about five hours every Friday night to through all of them. If you’re one of those folks who are gifted in math, you might have concluded that works out to about 72 sites an hour or about 1.2 a minute. You might be mildly curious how I do that. At least I hope you are, or this post isn’t going to keep your attention!
So if that’s not at all interesting to you, please feel free to bail!
Someday, I’ll have to give up programming. Until that day comes, I still enjoy an occasional foray into Java and IntelliJ.
The real point of the post isn’t the solution I came up with. It’s the idea that you have skills beyond writing. Maybe you draw or write programs. Maybe you host sock-puppet plays. I mention the last because I would love to see someone perform anime reviews with sock puppets.
Be that as it may, I used to write computer programs for a living. So, that’s the skill I brought to the blogging table! I hope that it’ll help you think about how you might approach some of the issues you’re facing.
360 Sites in 300 Minutes
When the Massive List of Anime Sites contained around 200 sites, getting through all of them in an evening was easy peasy. After I topped 300, though, I noticed that some of my Friday night sessions had started to expand into Saturday morning sessions. Since I’m trying to write a novel on weekends (plus Tuesdays!), I had to get a handle on that.
Now, I could have said, “Screw it! Random sampling wins the day!” If I check each site or randomly sample the sites, who would know? Well, I would know, and it would show in my writing. I’m stupidly honest enough to say something like, “I totally did not just sample the blog sites this week!” And then you’d trust me even less than now.
So I thought about the most time-consuming portion of the process. Or, more precisely, the most time-consuming portion that didn’t contribute directly to me reading the posts. I could save a ton of time not reading anything, but talk about defeating the purpose! As you might have guessed, the most time-consuming part of the process is going to the Massive List of Anime Sites, finding the next site to click on, clicking on it, moving to that tab, then reading the site. Then, I have to close the tab and find the next site. Rinse and repeat.
After the third hour, I was surprised how often I closed more than one tab. Or lost my place. So I started thinking about how I’d fix that.
Automating Blog Tasks: Sharing Ideas Helps
Do you remember when Irina from I Drink and Watch Anime used to put together weekly roundups of what blogs were reviewing what shows — like this post, Anime Winter 2019 – Week 9 Roundup!? Creating each of those posts consumed an enormous amount of time. So, trying to be helpful, I came up with a prototype batch or script file to automate opening the sites to see where the most recent review.
50 tabs looks like a lot, but you’d be surprised how fast you can get through them when you don’t have to open them one at a time!
The idea wasn’t bad, but it had a serious, fatal flaw: it had to be manually maintained. To be useful, you’d have to enter all of the sites into the code. After an initial investment, it might save a little time, but that initial investment really sucked. So I’m afraid it wasn’t useful. But I kept the idea in the back of my mind, because I thought it should be useful, if I could solve that issue.
Then it hit me: I already maintain the Massive List of Sites. I could capture the HTML code behind the Massive List of Anime Sites, feed it into a Java program, and create script files to open all 360+ sites for me!
Enter the Not Entirely Terrible Java Application
The workflow looks like this:
- I save the HTML source for Massive List of Anime Sites to a temporary directory on my iMac
- I edit that HTML so only the code for the unordered list of the active anime blogs remains
- Basically, I leave the <ul> tag and everything until the closing tag of </ul>
- I run my Java program
- The program looks at each entry in the unordered list, which are enclosed within list item tags <li></li>
- The program knows to carve the 360+ sites into manageable chunks of 50; that means a script for every 50 sites
- For each site, the program creates a script line to open the anime site in a separate browser tab
- The program leaves the individual scripts in a temporary directory, and I open a Terminal prompt to get to those scripts
- As I go through all 50 tabs looking for posts I like, I note my favorites in a text editor
- I keep running the next script until I run out of them
If you’re interested in the code, I’ve reproduced it below.
Astonishingly, It Works
The net effect is that I can go through all of the anime blog sites 50 at a time. I’ve found 50 is about the maximum before my iMac starts complaining. My iMac has 16Gb of RAM, which is usually enough. If I had less, I could adjust the code to create batches of 25 or something.
The intCount % 50 (mod) is what controls how many tabs each script will open. You could adjust it to 25 or 30 or whatever.
The code open tabs in Chrome; it should work as well for FireFox or Safari by changing the name of the brwoser. Here’s what the script looks like:
It’s a really simple approach, but it save me buckets of time.
What I like about this solution is that for the last several weeks, I’ve been able to get through all the sites in between 4 and 5 hours. Yes, I’m spending an average of 1.2 minutes per site, but more often that not, that’s enough to get a sense of what’s new. In fact, a large percentage of sites don’t publish even weekly. That means I get to that tab, see there’s nothing new, and close it.
The key idea isn’t the code. It’s the concept that there’s no telling how your skills other than writing might help you or your readers. I have a tendency to think in terms of boxes. This is my work box, so I code. This is my blog box, so I write. I thought a reminder that it’s okay to combine boxes might be helful!
Do you have some non-writing skills you use for your blog? If so, I’d love to hear about them in the comments!
The Java Code
If you’re interested, here’s the code. I didn’t use any libraries other than what’s in Java 1.8, so it should be easy to play with:
public class jobRunner
public void runJob(String strIputFile, String strOutputDirectory) throws Exception
File filInput = new File(strIputFile);
throw new Exception("Input file " + strIputFile + " does not exist");
File filOutDirectory = new File(strOutputDirectory);
throw new Exception("The directory " + strOutputDirectory + " does not exist.");
// See https://www.w3schools.com/java/java_files_read.asp
Scanner myReader = new Scanner(filInput);
List<String> list = new ArrayList<String>();
String strLine = myReader.nextLine();
String arrLines = strLine.split("<li");
for (int x = 0; x < arrLines.length; x++)
if (arrLines[x].indexOf("href") > 0)
Pattern p = Pattern.compile("href=\"(.*?)\"", Pattern.DOTALL);
Matcher m = p.matcher(arrLines[x]);
int intCount = 1;
FileWriter myWriter = new FileWriter(strOutputDirectory + "optca-" + Integer.toString(intCount) + ".sh");
for (int x = 0; x < list.size(); x++ )
String strLine = list.get(x);
myWriter.write("open -a \"Google Chrome\" \"" + strLine + "\"\n");
if (intCount % 50 == 0)
myWriter = new FileWriter(strOutputDirectory + "optca-" + Integer.toString(intCount) + ".sh");