Qiita Job I tried to analyze the job offer

table of contents

--Background --Data acquisition --I looked at the data --Impressions and future plans

background

While I was changing jobs, I was caught up in the phrase "Congratulations on changing jobs for 300,000 yen" and registered with Qiita Jobs, but when I tried using it, I thought that the functions were a little lacking.

I couldn't narrow down the work location, or I didn't know the number of pages.

Fortunately, I have most of the information I need in the job listings, and the total number of pages is only about 20 (obtained manually), so I decided to get it in Python.

Data acquisition

The general flow is like this.

--Get the content as HTML in the URL of the job list in Python --Create a beautiful soup with the acquired HTML, analyze tags, etc. and cut out the necessary parts. --The cut out part is converted to JSON and stored in Firebase cloud firestore. --On the other hand, save it locally as a csv file for analysis of takeover.

We got a total of 294 jobs. (Is this probably all?)

I looked at the data

The tools I used were Pandas and Tableau.

One thing I would like to say first is that the content from now on is created based on the content of the Qiita jobs job offer, so it will not reflect the job offer of the entire IT industry, but ** for your reference **.

The first thing I was most interested in was the number of job openings by work location.

work_place_num2.jpg

I see, the number of job vacancies in Tokyo has won overwhelmingly. Isn't it necessary to narrow down the work location? However, it seems a little inconvenient for those who are looking for a local job such as U / I turn.

Next, let's look at job vacancies by job type. 職種別求人数.jpg

TOP5 was a web application, other engineer, backend, frontend, and infrastructure engineer. I feel that the amount of work on the Web is more in demand than the work of smartphone development.

Also, to see what other engineers are looking for, I compared the back-end engineers and front-end engineers who have similar job openings to other engineers.

other engineer.jpg

As a result, for other engineers, there are a lot of technical tags such as JAVA, C ++, C #, games, animation, etc., probably embedded systems, game development, etc.

By the way, if you summarize the techniques used for all job offers, you will get the following hit map.

tagCount_heatMap.jpg

Finally, let's check the average annual income by job type, but it should be noted here that the "average annual income" is simply the average of the upper and lower limits of annual income.

職種別年収平均.jpg

Looking at this figure, you can see that the salary range of the web application engineer is large.

You can also see that the back-end engineer has a higher average annual salary than the front-end and back-end.

Impressions and future plans

This is the first time I've used Tableau, so I may still be unfamiliar with creating visualization diagrams, but I'd love to use it again if I have the opportunity.

I'm sure there's more you can do with the data you get from Qiita Jobs. I just searched the contents of the data today, but in the future

  1. I would like to use AWS services to create something that will see if new jobs are coming to the specified job category.
  2. This time, I used only the information that can be obtained from the list, but I think that it can also be used for machine learning because there is a lot of further information such as essential skills and working conditions in each job details.

In addition, you can check this drawing at the following URL. Tableau Public

Thank you for watching until the end.

Recommended Posts

Qiita Job I tried to analyze the job offer
I tried web scraping to analyze the lyrics.
I tried to move the ball
I tried to estimate the interval.
I tried to analyze the whole novel "Weathering with You" ☔️
I tried to summarize the umask command
I tried to summarize the graphical modeling.
I tried to estimate the pi stochastically
I tried to touch the COTOHA API
I tried to optimize while drying the laundry
I tried to save the data with discord
I started to analyze
[Qiita API] [Statistics • Machine learning] I tried to summarize and analyze the articles posted so far.
[Python] I tried to analyze the pitcher who achieved no hit no run
I tried to correct the keystone of the image
I tried to get the authentication code of Qiita API with Python.
I tried to debug.
(Python) I tried to analyze 1 million hands ~ I tried to estimate the number of AA ~
I tried to verify and analyze the acceleration of Python by Cython
I tried to paste
I tried to analyze the negativeness of Nono Morikubo. [Compare with Posipa]
LeetCode I tried to summarize the simple ones
I tried to analyze the New Year's card by myself using python
I tried to implement the traveling salesman problem
I tried to predict the price of ETF
I tried to vectorize the lyrics of Hinatazaka46!
I tried to graph the packages installed in Python
I tried to detect the iris from the camera image
I tried to summarize the basic form of GPLVM
I tried to touch the CSV file with Python
I tried to solve the soma cube with python
I tried to approximate the sin function using chainer
I tried to put pytest into the actual battle
[Python] I tried to graph the top 10 eyeshadow rankings
I tried to linguistically analyze Karen Takizawa's incomprehensible sentences.
I tried to visualize the spacha information of VTuber
I tried to erase the negative part of Meros
I tried to solve the problem with Python Vol.1
I tried to analyze J League data with Python
I tried to simulate the dollar cost averaging method
I tried to redo the non-negative matrix factorization (NMF)
I tried to identify the language using CNN + Melspectogram
I tried to notify the honeypot report on LINE
I tried to complement the knowledge graph using OpenKE
I tried to classify the voices of voice actors
I tried to compress the image using machine learning
I tried to summarize the string operations of Python
I tried to organize SVM.
I tried to implement PCANet
I touched the Qiita API
I tried the changefinder library!
I tried to reintroduce Linux
I tried to introduce Pylint
I tried to summarize SparseMatrix
I tried to touch jupyter
I tried to implement StarGAN (1)
I tried to find the entropy of the image with python
I tried to find out the outline about Big Gorilla
I tried to introduce the block diagram generation tool blockdiag
I tried porting the code written for TensorFlow to Theano
[Horse Racing] I tried to quantify the strength of racehorses