2023-12-24-111303 Systematically identifying skill sets relevant to data science - the hard way

[2023-12-24 Sun 11:13]

SocialOutreach MyProjects? OddmuseWiki DataScienceJournal CareerJournal BusinessJournal

I run a little data science jobs wiki. It is an OddmuseWiki in which I manually collect job descriptions that are of interest to me. The jobs are obviously related to DataScience and MachineLearning? in general.

I've written a bit about that wiki in the past, at a time when I was not regularly updating it or looking at it. The conclusion at the time was that it still seemed useful. This is a review and updated version of whether I think it is still useful.

The exercise of publishing that wiki started with encountering the Data Science Weekly Guide which includes a method to systematically build a profile and identifying the necessary skillsets for a successful job application in data science.

The gist of the approach is that, the only tangible source of information with respect to a job is the job posting. As most experienced people know, the profile usually does not match perfectly with the actual job. It is a lot worse for Data science whererin the typical expectation seems to be that of a DataUnicorn??. However, the job profile on an average is still a better approximation than an uneducated, personal guess of the skills and knowledge needed to find a job in Data Science. In the least it reflects the desires of the posting company irrespective of reality.

In a rather simplified statement: The Data Science weekly authors talk about collecting job descriptions in a high number (preferably >100) and then filtering them out in a binary manner with the question “Do I find this interesting?” with only the information available at hand, and spending a very short amount of time. Thus one is left with 2 piles, after which the excercise is repeated unless one is left with barely 10 companies or job profiles. This has pleasant connections

This approach seemed to make a lot of sense to me, especially as I transitioned to data scicence from a different field, and because of the large variety in the career advice that one tends to encounter in practice (which in hindsight is not all that surprising).

Along the way, considering that I was already building an personal, local wiki using Emacs and OrgMode??, I thought it might be cool to post this online, as it could be useful to other folks.

Coming down to brass tacks, what do I think are the pros and cons of continuing it today?

• I have my own public list of recruiting agencies in Canada, and can quickly refer to this list in a pinch, or leisurely when I want to hunt for a job. It's not just a simple list, but also an interlinked wiki with additional and information useful in varying degrees.
• I have an additional (growing) taxonomy, of varying kinds like manufacturing, fintech, insurance and also related to company sizes, remote working, salary indication and more importantly 'Profiles and companies that are interesting to me'. The last bit is very aligned to the exercise suggested by Data Science weekly publication. I have found this taxonomy strangely fulfilling in the sense that it helps me identify where I am and the potential characteristics of differences between companies, even if job profiles look the same.
• The effort to plug in a job profile, company name and so on are reasonably streamlined in the sense, it does not take too long to be noticeably annoying. I have mostly shifted to directly editing the wiki with job descriptions rather than maintaining a local org mode version. There are only some basic formatting changes to be applied, and there are handy OrgMode? commands already available to do so.
• One clear con is about the information becoming outdated. Startups may fail, the facts like number of employees may change drastically and so on.
• Another clear con is that none of this is exciting or novel enough to share with anybody with 'a sense of pride'. It's just another sad, lonely wiki. Practically, when push comes to shove - you login to a job portal and just apply. However, despite that - I have grown to like my little wiki of information.
• I have a reasonable method to track the changes in job profiles and requirements over time. By forcing myself to collect say 1-2 job descriptions a week, I can keep myslf upto date on current expectations and general industry knowledge. It is interesting to discover new companies and try to quickly understand what they do. I think it is even critical in a way, as a method to not become jaded.
• There is always a potential that I can harvest all the information on the wiki and feed it to an LLM or something else to extract more insight.
• A clear con is that it is somewhat futile and boring to list every company out there. Over time, I've become more and more selective, and it is important that I also focus on tending to the garden, in the sense, gathering more information about the companies that I have found rather than just adding more.

I could go on, but it seems that the pros outweigh the cons in general. There are some dangers here, in the sense of mechanically collecting information for the sake of it without processing and accumulating 'wisdom'. I think my aversion to manual work will help me stay on track in this respect while I keep the IkeaEffect in mind.

It is not a 'sexy project' and perhaps not one that I may list on my resume (today). However, especially in the recent past - and whenever I feel like I want to know what is out there - I increasingly tend to take a look at the jobs wiki rather than a job portal.