最资讯丨使用Python爬取股票数据方法分析

使用Python爬取股票数据目标流程为: 获取证券代码、证券简称、交易日期、开盘价、最高价、最低价、收盘价、成交数量；

(资料图片)

难点: 请求头参数mcode 加密生成；

使用第三方库:

requests

execjs

js2py

math

time

pandas

工具:

谷歌浏览器

pycharm

python3.7

爬取到的股票数据存入到code.xlsx，可设定需要爬取的时间范围。

具体爬取数据程序如下：

import requests

import execjs

import js2py

import math

import time

import pandas as pd

code_list = []

def MCODE():

jscode = """

function missjson(input) {

var keyStr = "ABCDEFGHIJKLMNOP" + "QRSTUVWXYZabcdef" + "ghijklmnopqrstuv" + "wxyz0123456789+/" + "=";

var output = "";

var chr1, chr2, chr3 = "";

var enc1, enc2, enc3, enc4 = "";

var i = 0;

do {

chr1 = input.charCodeAt(i++);

chr2 = input.charCodeAt(i++);

chr3 = input.charCodeAt(i++);

enc1 = chr1 >> 2;

enc2 = ((chr1 & 3) << 4) | (chr2 >> 4);

enc3 = ((chr2 & 15) << 2) | (chr3 >> 6);

enc4 = chr3 & 63;

if (isNaN(chr2)) {

enc3 = enc4 = 64;

} else if (isNaN(chr3)) {

enc4 = 64;

}

output = output + keyStr.charAt(enc1) + keyStr.charAt(enc2)

+ keyStr.charAt(enc3) + keyStr.charAt(enc4);

chr1 = chr2 = chr3 = "";

enc1 = enc2 = enc3 = enc4 = "";

} while (i < input.length);

return output;

}

"""

time1 = js2py.eval_js("Math.floor(new Date().getTime()/1000)")

# py方式

a = math.floor(time.time() / 1000)

mcode = execjs.compile(jscode).call("missjson", "{a}".format(a=time1))

return mcode

def PageRquest(datetime, mcode):

# 接口可以换

url = "http://webapi.cninfo.com.cn/api/sysapi/p_sysapi1015"

data = {

"tdate": datetime, # 获取数据时间

"scode": "399001" # 股票代码 以及交易所简称

}

headers = {

"mcode": str(mcode),

"Referer": "http://webapi.cninfo.com.cn/",

"Cookie": "Hm_lvt_489bd07e99fbfc5f12cbb4145adb0a9b=1634795282; Hm_lpvt_489bd07e99fbfc5f12cbb4145adb0a9b=1634799860",

"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.54 Safari/537.36"

}

response = requests.post(url, headers=headers, data=data).json()

code = response["records"]

for i in code:

code_list.append(i)

def main(date):

mcode = MCODE()

PageRquest(date, mcode)

if __name__ == "__main__":

# main()

# 数据分析 pandas 自动化办公的

datetime = pd.period_range("2021/5/26", "2021/10/27", freq="B")

for date in datetime:

main(date)

df = pd.DataFrame(code_list)

print(df)

df.to_excel("code.xlsx")

责任编辑：