I tried to aggregate & compare unit price data by language with Real Gachi by Python

I wanted to compare and aggregate the unit prices of freelance projects by language from before, so I actually investigated it.

Information on freelance projects is aggregated based on the following site Since there was aggregated information on the page of each language, it is based on that

Revatec Freelance https://freelance.levtech.jp

** * Data as of November 21, 2020 **

Cloud.csv
created_at,skill,count,avg_price,max_price,min_price
20201121,OpenShift,5,62,65,60
20201121,Amazon VPC,1,70,70,70
20201121,Google Cloud Platform,55,80,105,53
20201121,Dynamics CRM,1,65,65,65
20201121,Amazon S3,2,80,85,75
20201121,Amazon EC2,2,74,85,64
20201121,AWS,1926,76,135,20
20201121,Google App Engine,7,77,95,60
20201121,Amazon SimpleDB,1,80,80,80
20201121,Force.com,2,70,70,70
20201121,Office 365,9,56,65,40
20201121,Microsoft Azure,141,67,90,50
20201121,OpenStack,21,64,80,45
20201121,Heroku,29,75,102,55
DB.csv
created_at,skill,count,avg_price,max_price,min_price
20201121,Kyoto Tycoon,1,75,75,75
20201121,SQLite,20,72,90,55
20201121,MySQL,1465,75,125,35
20201121,Symfoware Server,4,60,65,55
20201121,Cassandra,14,62,80,45
20201121,PostgreSQL,487,71,135,35
20201121,Riak,5,63,100,45
20201121,Bigtable,3,86,95,80
20201121,SQL Server,375,65,95,35
20201121,Redis,296,78,105,39
20201121,Hbase,5,68,75,60
20201121,Oracle,971,66,115,25
20201121,Sybase,6,75,85,65
20201121,IMS,3,58,64,55
20201121,Access,88,60,145,40
20201121,DB2,79,64,95,35
FrameWork.csv
created_at,skill,count,avg_price,max_price,min_price
20201121,Pyramid,12,77,105,65
20201121,Rails,237,81,115,45
20201121,MyBatis,61,71,85,55
20201121,Node.js,256,77,125,45
20201121,Backbone.js,77,76,115,39
20201121,Knockout.js,4,72,90,55
20201121,AngularJS,152,74,105,47
20201121,Laravel,343,73,125,40
20201121,JUnit,80,73,125,50
20201121,Wicket,3,82,95,70
20201121,Django,89,83,115,55
20201121,Padrino,2,82,85,80
20201121,iBATIS,14,67,80,55
20201121,Silex,4,76,80,75
20201121,FuelPHP,101,76,115,50
20201121,MVC,69,68,95,37
20201121,CodeIgniter,39,74,105,45
20201121,Liferay,1,70,70,70
20201121,React,378,77,115,40
20201121,Spark,19,82,95,55
20201121,PhoneGap,3,86,95,80
20201121,JSF,18,64,75,55
20201121,jQuery,354,71,95,35
20201121,Seasar2,32,70,95,49
20201121,Spring,362,71,115,45
20201121,Bottle,1,75,75,75
20201121,Catalyst,26,64,72,55
20201121,intra-mart,14,70,135,50
20201121,SAStruts,16,71,95,57
20201121,Flask,25,82,125,65
20201121,Sinatra,6,78,85,75
20201121,Struts,129,67,95,45
20201121,Symfony,55,74,95,55
20201121,CakePHP,149,71,95,50
Language.csv
created_at,skill,count,avg_price,max_price,min_price
20201121,PHP,1560,72,145,35
20201121,CSS,650,70,125,35
20201121,C language,269,68,115,35
20201121,UML,7,70,85,50
20201121,C++,374,72,125,45
20201121,C#.NET,147,62,80,32
20201121,Transact-SQL,6,65,75,55
20201121,Kotlin,270,83,125,45
20201121,VB,83,62,85,40
20201121,HTML5,305,72,115,35
20201121,VC,8,61,66,50
20201121,C#,706,68,115,35
20201121,Apex,49,79,135,55
20201121,VBA,165,60,145,30
20201121,ASP.NET,161,65,95,48
20201121,Go,361,81,125,39
20201121,PL/SQL,146,63,85,45
20201121,COBOL,81,61,95,37
20201121,LESS,9,74,115,55
20201121,Swift,407,81,125,50
20201121,VC++,30,65,90,52
20201121,HTML,674,69,125,35
20201121,CSS3,209,75,115,35
20201121,Java,2484,70,145,25
20201121,JSP,54,68,105,55
20201121,Sass,55,71,115,45
20201121,SQL,881,65,145,33
20201121,VBScript,44,62,75,45
20201121,Objective-C,247,76,115,40
20201121,CoffeeScript,16,81,95,65
20201121,JavaScript,1749,72,125,20
20201121,Lua,6,73,80,65
20201121,R language,19,78,95,65
20201121,Smalltalk,2,75,80,70
20201121,Scala,117,83,115,55
20201121,Perl,108,73,110,50
20201121,Shell,193,64,95,45
20201121,SAS,12,61,75,50
20201121,Ruby,652,80,125,39
20201121,ABAP,12,67,80,55
20201121,Python,669,78,145,32
20201121,VB.NET,284,62,85,40
OS.csv
created_at,skill,count,avg_price,max_price,min_price
20201121,Windows,998,63,135,25
20201121,Red Hat,92,66,95,40
20201121,Solaris,37,61,80,40
20201121,AIX,77,63,90,52
20201121,Ubuntu,39,75,90,45
20201121,Unix,236,69,100,40
20201121,AS/400,1,65,65,65
20201121,FreeBSD,3,70,75,62
20201121,HP-UX,21,62,75,55
20201121,Windows Server,387,62,100,33
20201121,Android,563,78,125,35
20201121,Linux,2266,69,135,30

Click here for graph image output source

print_graph.py

import datetime
import os
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

"""
Common variable / constant setting
"""
DATA_PATH = 'data'
now = datetime.datetime.today()
CATEGORIES = {
    'Language',
    'FrameWork',
    'DB',
    'OS',
    'Cloud'
}

"""
Main processing
"""
def main() -> None:

    make_date = now.strftime("%Y%m%d")

    #Directory creation
    os.makedirs(DATA_PATH + '/graphs/' + make_date, exist_ok=True)

    price_graph(make_date)
    count_graph(make_date)
    mix_graph(make_date)


"""
Unit price graph creation
"""
def price_graph(make_date: str) -> None:

    for category_key in CATEGORIES:
        df = pd.read_csv(DATA_PATH + '/levtech/' + category_key + '.csv')
        df = df.sort_values(by=['avg_price'], ascending=False)
        labels = df['skill'].str.replace('[Ah-Hmm-One-龥]+', '', regex=True)  #Delete Japanese
        y_list1 = df['avg_price']

        fig = plt.figure(figsize=(14, 7))

        ax1 = fig.add_subplot(1, 1, 1)
        ax1.bar(labels, y_list1, color='tab:blue')
        ax1.set_ylabel('price')
        ax1.set_title(category_key)

        #Vertical display of X axis
        ax1.set_xticklabels(labels, rotation=90)

        #Add numerical value to bar graph
        for x, y in zip(labels, y_list1):
            ax1.text(x, y, y, ha='center', va='bottom')

        # plt.show()
        fig.savefig(DATA_PATH + '/graphs/' + make_date + '/levtech-' + category_key + '-price.png', bbox_inches='tight', format='png', dpi=300)


"""
Create a graph of the number of cases
"""
def count_graph(make_date: str) -> None:

    for category_key in CATEGORIES:
        df = pd.read_csv(DATA_PATH + '/levtech/' + category_key + '.csv')
        df = df.sort_values(by=['count'], ascending=False)
        labels = df['skill'].str.replace('[Ah-Hmm-One-龥]+', '', regex=True)  #Delete Japanese
        y_list1 = df['count']

        fig = plt.figure(figsize=(14, 7))

        ax1 = fig.add_subplot(1, 1, 1)
        ax1.bar(labels, y_list1, color='tab:orange')
        ax1.set_ylabel('count')
        ax1.set_title(category_key)

        #Vertical display of X axis
        ax1.set_xticklabels(labels, rotation=90)

        #Add numerical value to bar graph
        for x, y in zip(labels, y_list1):
            ax1.text(x, y, y, ha='center', va='bottom', fontsize='small')

        # plt.show()
        fig.savefig(DATA_PATH + '/graphs/' + make_date + '/levtech-' + category_key + '-count.png', bbox_inches='tight', format='png', dpi=300)


"""
Unit price & number of projects graph creation
"""
def mix_graph(make_date: str) -> None:

    for category_key in CATEGORIES:
        df = pd.read_csv(DATA_PATH + '/levtech/' + category_key + '.csv')
        df = df.sort_values(by=['avg_price'], ascending=False)

        labels = df['skill'].str.replace('[Ah-Hmm-One-龥]+', '', regex=True)  #Delete Japanese
        y_list1 = df['avg_price']
        y_list2 = df['count']

        fig = plt.figure(figsize=(14, 7))

        left = np.arange(len(labels))
        width = 0.3

        ax1 = fig.add_subplot(1, 1, 1)
        ax1.bar(labels, y_list1, width=width, color='tab:blue', label='price')
        ax1.set_ylabel('price')
        ax1.set_title(category_key)

        ax2 = ax1.twinx()
        ax2.bar(left + width, y_list2, width=width, color='tab:orange', label='count')
        ax2.set_ylabel('count')

        #Vertical display of X axis
        ax1.set_xticklabels(labels, rotation=90)

        #Add numerical value to bar graph
        for x, y in zip(labels, y_list1):
            ax1.text(x, y, y, ha='center', va='bottom')

        for x, y in zip(left + width, y_list2):
            ax2.text(x, y, y, ha='center', va='bottom', fontsize='small')

        #Output legend
        handler1, label1 = ax1.get_legend_handles_labels()
        handler2, label2 = ax2.get_legend_handles_labels()
        ax1.legend(handler1 + handler2, label1 + label2)

        # plt.show()
        fig.savefig(DATA_PATH + '/graphs/' + make_date + '/levtech-' + category_key + '-mix.png', bbox_inches='tight', format='png', dpi=300)


if __name__ == '__main__':
    main()

Actual graph

--Price is the average unit price (10,000 yen) --count is the number of cases ――The graph in which the unit price and the number of projects are mixed is in descending order of unit price.

language

By unit price

levtech-Language-price.png

By number of cases

levtech-Language-count.png

Unit price and number of projects mix

levtech-Language-mix.png

The top is Kotlin. I'm convinced that I hear that there is a shortage of Android engineers now. If you look at the number of projects and the unit price for the web system, Ruby, Python, and go seem to be good for cospa. I had the impression that Java had a high unit price for the time being, but it is surprising that PHP is more expensive ... Is the demand too high?

Then the framework

By unit price

levtech-FrameWork-price.png

By number of cases

levtech-FrameWork-count.png

Unit price and number of projects mix

levtech-FrameWork-mix.png

The top is PhoneGap. I heard it for the first time, but it seems to be a cross-platform mobile application development framework developed and released by Nitobi. If it's a web system, Django and Rails seem to be good at cospa. Since React is used for web and application development, the number of projects and unit price are good. Vue had no information at the site source. ..

DB

By unit price

levtech-DB-price.png

By number of cases

levtech-DB-count.png

Unit price and number of projects mix

levtech-DB-mix.png

The top is Bigtable. I often hear Bigquery, but the difference seems to be like a large DB that runs faster than Bigquery. Regarding DB, there is no doubt if you remember MySQL in terms of cost performance. Oracle and SQL Server are surprisingly low. why,,,

Cloud

By unit price

levtech-Cloud-price.png

By number of cases

levtech-Cloud-count.png

Unit price and number of projects mix

levtech-Cloud-mix.png

The top unit price is GCP! AWS is overwhelming when it comes to the number of projects! For the time being, it seems better to remember AWS at least. If the demand becomes too high, the unit price tends to decrease, so considering the future, it may be better to remember GCP as well. Azure has a delicate number of deals and unit prices

Impressions

The result was quite interesting! If you do web-based things from now on, Python may grow. It's easy to build an environment by actually touching it. Rich library. I was particularly impressed with how easy it is to create CSV → graphs! It's really nice to be able to quickly create and image with the back end rather than using the html or js library to go crazy ~

This time, it is the result of investigating one of the project sites, so please use it as a reference! Information on Vue, Graphql, and Rust wasn't listed, so I'm worried about that.

Recommended Posts

I tried to aggregate & compare unit price data by language with Real Gachi by Python
I tried to get CloudWatch data with Python
I tried to analyze J League data with Python
I tried to make various "dummy data" with Python faker
I tried a functional language with Python
[Data science basics] I tried saving from csv to mysql with python
I tried to communicate with a remote server by Socket communication with Python.
How to write offline real time I tried to solve E11 with python
I tried fMRI data analysis with python (Introduction to brain information decoding)
How to write offline real time I tried to solve E12 with python
I tried to program bubble sort by language
I tried to save the data with discord
I tried to output LLVM IR with Python
I tried to automate sushi making with python
I tried to open the latest data of the Excel file managed by date in the folder with Python
I tried to implement Minesweeper on terminal with python
I tried to get started with blender python script_Part 01
I tried to touch the CSV file with Python
I tried to draw a route map with Python
I tried to solve the soma cube with python
NW engineer tried to aggregate addresses with python netaddr
I tried to implement an artificial perceptron with python
I tried to solve the problem with Python Vol.1
I tried to solve AOJ's number theory with Python
I tried to compare the processing speed with dplyr of R and pandas of Python
I tried scraping food recall information with Python to create a pandas data frame
I tried fp-growth with python
I tried scraping with Python
I tried gRPC with Python
I tried scraping with python
I want to be able to analyze data with Python (Part 3)
I tried various methods to send Japanese mail with Python
I want to be able to analyze data with Python (Part 1)
I want to be able to analyze data with Python (Part 4)
[Introduction to Pandas] I tried to increase exchange data by data interpolation ♬
[Python] I tried to visualize tweets about Corona with WordCloud
I tried to classify MNIST by GNN (with PyTorch geometric)
Mayungo's Python Learning Episode 3: I tried to print numbers with print
I tried to predict Boston real estate prices with PyCaret
I tried to make GUI tic-tac-toe with Python and Tkinter
I tried to divide the file into folders with Python
I tried to divide with a deep learning language model
How to write offline real time I tried to solve the problem of F02 with Python
I tried to describe the traffic in real time with WebSocket
I tried to solve the ant book beginner's edition with python
I tried to touch Python (installation)
[3rd] I tried to make a certain authenticator-like tool with python
[Python] A memo that I tried to get started with asyncio
I tried to verify the speaker identification by the Speaker Recognition API of Azure Cognitive Services with Python. # 1
Stock price plummeted with "new corona"? I tried to get the Nikkei Stock Average by web scraping
I tried to fix "I tried stochastic simulation of bingo game with Python"
I tried to make a periodical process with Selenium and Python
I tried to find out if ReDoS is possible with Python
I tried to make a 2channel post notification application with Python
I tried to implement sentence classification by Self Attention with PyTorch
I tried to verify the speaker identification by the Speaker Recognition API of Azure Cognitive Services with Python. # 2
I tried to summarize everyone's remarks on slack with wordcloud (Python)
[4th] I tried to make a certain authenticator-like tool with python
I tried to easily detect facial landmarks with python and dlib
[1st] I tried to make a certain authenticator-like tool with python
I tried to improve the efficiency of daily work with Python