Automating a job search part 1

At no point do I guarantee usable results, even if you follow the instructions, all I can say is it worked for me, I also don’t suggest that any of this wont cause unintended data leakage, and I don’t imply that this is within local laws, guidelines, TOS agreements, or anything like that. If you do this perfectly it may still catch on fire or yell obscenities at grandma, but thats pretty unlikely.

This guide is designed to help you create a bot that can be used to archive RSS items to an email address, it does not do any real “web scraping”, but can be a great way to keep updated about gigs, as you read through, you’ll see other ways you can use this to archive almost anything. The drawbacks of this simple method are a few main things, RSS feeds are usually truncated, which means you may have to click through to read the full content, additionally, since this article is aimed at job hunting, the other big drawback is that it won’t give you contact information in the feed, like email or phone numbers. I’ll write up another article for shimming in the tools which will help us extract that information in the near future. Also, since we’re basically copying text from the RSS feed, we won’t be notified if the content changes, or when it goes away. This article is intended for research use so you can get into playing with RSS feeds, and is not intended for private or production use. If you are using it “for profit” you will likely be violating TOS and you generally should not rely on free services, as they may change how they work without notification.

This is a beginner level automation project, It was the first stage of a “bot” I’d written quite a long time ago, the finished bot included a lot more, a service called yahoo pipes (which is gone now) lots of regex and the end result was a bot that generated cover letters and automatically applied for jobs by email, it also attached a PDF generated version of my current resume as was found on linkedin, pipes is gone now, but as part of this series I’ll try to recreate the whole thing, this (part 1) requires very little technical knowledge, and as you’ll see, at the first stage requires no code be written at all. You need an email account for your bot, a basic understanding of using search filters, and an account at ifttt. Essentially, ifttt is the motor for this, and it works by getting permission to access your accounts, you tell it to do things based on events, you may not want ifttt to have unfettered access to your personal accounts, so it may be a better choice to use a dedicated “robot” email account I recommend gMail because of its “other” account features which we will be taking advantage of down the road.

We’ll start with is the Gigs section in Craigslist. We’ll be using search operators to reduce the junky-ness of the results, as you’ll see, the blacklisting of certain terms only offers an improvement, definitely not a final result. I’ve already set a search string, common words or pairings that generally look spammy so they wont be included in our results. Again, this is the “blacklisting” approach, which means removing stuff we think is bad, a better approach is to combine it with whitelisting (only include results that meet a certain requirement), which will get us usable results. Here’s the link for the search with Blacklisting, and heres a link with the search combined with whitelisting,, this adds lawn and grass so the results will only include fewer spamishly written posts which also include lawn or grass, it should default to the most recent city you used.
Work on that a bit, unless your looking for lawn cutting jobs nearby. The next step is to make it notify you, in this case we’ll be using CL’s RSS feature and IFTTT to send us an email.

Across the bottom of the page, you should see something like this,

Click the orange RSS icon on your own search results on the craigslist page. That will take you to a page called an RSS Feed, its not the prettiest, but you only need to look at it long enough to verify your results. Here’s my generated link, if you look closely at the url and your into scripting, you’ll notice it could be fun to integrate into something that automates creation of contextual searches by modifying the URL, thats another rabbit hole we’ll be exploring later.

On to ifttt, Setup your account, as I’d said earlier, you may want to use a dedicated bot account, if you haven’t yet created an email account for your bot, go ahead and do that, or use your personal account. Again, I don’t recommend using your personal email/gmail account. So log into ifttt…

Click your name/username on the top right.
New Applet
Click the “+”(plus) Sign, next to the word “This”
Type “feed” in the searchbar, click feed.
Paste the Craigslist RSS url we created earlier and submit
Click the “+that”
Email, send yourself an email, and you can leave the default settings.
Your pretty much done at that point. There are some improvements we can make.

Find our free private WordPress hosting guide, Email the posts to that wordpress page. If its publicly accessible, then you may be violating the TOS.

Forward it to your phone via text message (or sign in to the email account) so you can get notifications immediately about new jobs that fit what your looking for.

Use some web scraping techniques to grab the full text of the post, and extract contact information, we’ll see that this has LOT’s of caveats, and can be quite time consuming and may require regular maintenance when one of your 10’s or hundreds of source sites makes changes to their formatting.

As we work through the automated job robot, next we’ll look into integrating other job board type sites, we’ll start with a list of easier ones which can email you. Well take those emails and run them through some processing/filtering to make sure they actually fit *you*, and then well look at integrating more complex sites, like job boards without RSS feeds.

Lastly, we could may a scoring system, that way when a perfect job gets posted you get an alert with extra enthusiasm. We can look into generating a (kind of) unique cover letter, that gets deep, it can be as unique as you want, but you’ll see as it gets more complex it gets exponentially more complex, which is why those automatic generated emails usually look like crap, ours will be better. And we’ll look into automatically attaching a fresh copy of our resume. Once the whole thing is complete, if you get it just right, you’ll might be able to simply sit back and wait for the calls of employers who might like to meet you.

Any images are owned by their respective companies, the gmail logo was taken from the gmail wikipedia page, craigslist was a screenshot of the page, and ifttt logo was taken from a screenshot of the ifttt page. Craigslist results images are from craigslist.

Free technical learning resources

I’m really not sure how to organize this, lots of overlap, everywhere. These will most likely all get broken into individual posts for easier searching.
JavaScript/Web design
odin project  Free, open source based full stack training program. – Javascript, html, D3, CSS, React, Node, Git & Github, more.
Hackerbay  Work on real projects and gain skills in “full stack” development.
Youtube/Video series
Anything with Andrew Krouthamel (Networking Hardware, Firewalls)
SirNetrom1’s My Playhouse – Datacenter in the basement hijinks
Byte My Bits – Some homelab stuff
UbuntuGuy – Homelab stuff And UbuntuGuy’s other list
professormesser – Older stuff, but very high quality.
INEtraining has a paid site below.
Pascom Asterisk Playlist Well recommended playlist for Asterisk.
VMware Youtube Series
ITFreeTraining looks like it maybe needs some TLC at the time of writing
Eli the Computer Guy (lots of basics)
Not yet organized in a subgroup linux basics
Deployment Research (Windows deployment mostly)
Harvard (Not really well organized imho)
Ms TestLabs Guides

vmware more labs. (hol?)

Class Central – lots of stuff, information technology section linked

Code Academy

VMware MyLearn

Comprehensive (two or more of videos, documents, tests, forums, etc)
Professor Messer (seems outdated, but this stuff does not change *that* fast)
Cisco  learning network
Windows 10 virtual labsAnother link, and another link At the time of writing they look a bit different, will care to investigate later.
Windows MVA for PowerShell
Linux Shells (List of free shells, a few allow sudo, usually after you jump through some hoops like sending a postcard)
Amazon Offers a free year on the AWS platform with their version of linux.

IPspace has a *free* tier, looks promising
Cybrary – IT Certifications video training
Security Tube hacking/security(I think this is less maintained lately)
Free College Classes
MIT (Already filtered for IT related stuff)
Not strictly technical
You can (probably) get access to through your local public library.
Good free book sources
Redhat Technical Books (PDF’s)

Paid options… (these get surprisingly pricey, but most offer Free Trials) is gaining popularity lately, offers a free trial.
cbtnuggets is one of the most commonly recommended IT learning sites.
Linux Academy
Oreilly has one, great books, training is probably good
IPspace has a free *tier* so it got mentioned above as well.

About me

So, I figured I’d share some of the things I’m working on. This post will be occasionally updated, I’ll make it into a “page”. Also, this post is all about me. Really this whole site is about me, I’m hoping to get a few friends and like minded people involved, so hopefully this will change soon.
The reason… “look, feel, and purpose”
I like Open source and free stuff, I don’t have a budget so most of what I write will be free, cheap, or “The necessary tool” for the job.
As for budget, I’ll be hosting limited advertisements here, at this point the site only costs me money. I understand that you don’t want my ads, if you like the content please consider throwing some change at me or something.
Old reposts, there is some stuff on the web I love to re-read, like the worlds worst hacker, I’ll be reposting non original content on some occasions. I also have some old content I’d written in the past, I will bring some of it over here.
I write scripts, very often. Computer scripts, VB, batch, bash, powershell, etc. I want to share some of the more complex stuff I write for network management. I’ll also be adding in some spiffy automation kits, using things like AutoItV3 (windows Macros), iMacros (Firefox/Chrome), xDoTool (linux Macros), along with Fake (Web automation for OS X), and automator (OS X).
I make websites, its a hobby I am not very great at, I still do it for some reason. I write HTML, CSS, PHP, MySql, and JavaScript. I’m studying Ajax, but that seems to mostly mean “clever ways to use JavaScript”.
I manage networks, Mostly windows, but I some with Macs, along with windows and linux servers. I’ll scribble stuff about all that here too.
I’m learning about Audio and Video production and editing. Right now I am working with OBS as well as Blender, those are mostly video, I’ll be doing some stuff with recording plain audio as well as recording a band or group.

That really sums up what I “do”, As for some upcoming posts…

Pi Projects (these could also be called “linux” projects, but the pi’s have limitations and features that make them a little different, mostly slower and smaller)
Web Server with nGinx  & WordPress
Access point/Repeater
Network Diagnostics toolkit (with automation)

I’m really wanting do get in to additive and reductive construction, in “normal folk words” that means 3d printing and CNC.
Scripts! I’m working on some absurdly complex scripts.
Basics I see so many people trying to learn the latest and greatest stuff in technology while ignoring the basics, so I’ll try to spread some boring knowledge about basics.
 How to set up networks, stuff about pfsense, packet filter, OSX and Windows firewall, iptables, Sonicwall and Ubiquiti shenanigans. And more.
Just for fun Tips for writing scripts better, packers, unpackers, command line games, loading up Amazon Fire sticks with extra apps to expand functionality.
Down the road I’ll hopefully be adding videos, a podcast, and lots of pictures.
Thanks for reading, hopefully this wasn’t at the top by the time you got here.