采集app天天快报相关资讯

app上的资源，PC端看不到。只能用抓包工具，来抓包数据源。方法：安装安卓虚拟机，打开天天快报app，利用fiddler抓包。抓取URL：以搜索“音乐”为例。http://r.cnews.qq.com/search?uid=1815&store=274&hw=iPhone_7plus&devid=868857451279719&appversion=2.9.0&screen_width=576&mac=08%3A00%3A27%3Ad1%3A36%3A72&appver=17_areading_2.9.0&android_id=1815&origin_imei=868857451279719&sceneid=&mid=d39d21efb40861ec456ba62ace11fad&apptype=android&screen_height=1024&qqnetwork=wifi&omgbizid=73cf38b05c424f459dbaa6fd6fcbb5bd288db&Cookie=%20lskey%3D%3B%20luin%3D%3B%20skey%3D%3B%20uin%3D%3B%20logintype%3D0%3B&qn-rid=7dd2718a-a6b5-40be-ae94-8e23e0836d21&imsi_history=463&query=音乐&qn-sig=5bfecb6e5758e1f6b3ff9acb85da842a&activefrom=icon&imsi=463&omgid=cf268fc07aedcc46c33a7b078a18a2b9cd36b

liunix环境下输入命令：# curl -s 'http://r.cnews.qq.com/search?uid=1815&store=274&hw=iPhone_7plus&devid=868857451279719&appversion=2.9.0&screen_width=576&mac=08%3A00%3A27%3Ad1%3A36%3A72&appver=17_areading_2.9.0&android_id=1815&origin_imei=868857451279719&sceneid=&mid=d39d21efb40861ec456ba62ace11fad&apptype=android&screen_height=1024&qqnetwork=wifi&omgbizid=73cf38b05c424f459dbaa6fd6fcbb5bd288db&Cookie=%20lskey%3D%3B%20luin%3D%3B%20skey%3D%3B%20uin%3D%3B%20logintype%3D0%3B&qn-rid=7dd2718a-a6b5-40be-ae94-8e23e0836d21&imsi_history=463&query=音乐&qn-sig=5bfecb6e5758e1f6b3ff9acb85da842a&activefrom=icon&imsi=463&omgid=cf268fc07aedcc46c33a7b078a18a2b9cd36b'|jq .// 大家可以看到数据源了，我们只需要URL。

采集URL，输入命令：#curl -s 'http://r.cnews.qq.com/search?uid=1815&store=274&hw=iPhone_7plus&devid=868857451279719&appversion=2.9.0&screen_width=576&mac=08%3A00%3A27%3Ad1%3A36%3A72&appver=17_areading_2.9.0&android_id=1815&origin_imei=868857451279719&sceneid=&mid=d39d21efb40861ec456ba62ace11fad&apptype=android&screen_height=1024&qqnetwork=wifi&omgbizid=73cf38b05c424f459dbaa6fd6fcbb5bd288db&Cookie=%20lskey%3D%3B%20luin%3D%3B%20skey%3D%3B%20uin%3D%3B%20logintype%3D0%3B&qn-rid=7dd2718a-a6b5-40be-ae94-8e23e0836d21&imsi_history=463&query=音乐&qn-sig=5bfecb6e5758e1f6b3ff9acb85da842a&activefrom=icon&imsi=463&omgid=cf268fc07aedcc46c33a7b078a18a2b9cd36b'|jq '.'|grep '\'url\':'

稍微处理下数据，得到纯净的URL。输入命令：curl -s 'http://r.cnews.qq.com/search?uid=1815&store=274&hw=iPhone_7plus&devid=868857451279719&appversion=2.9.0&screen_width=576&mac=08%3A00%3A27%3Ad1%3A36%3A72&appver=17_areading_2.9.0&android_id=1815&origin_imei=868857451279719&sceneid=&mid=d39d21efb40861ec456ba62ace11fad&apptype=android&screen_height=1024&qqnetwork=wifi&omgbizid=73cf38b05c424f459dbaa6fd6fcbb5bd288db&Cookie=%20lskey%3D%3B%20luin%3D%3B%20skey%3D%3B%20uin%3D%3B%20logintype%3D0%3B&qn-rid=7dd2718a-a6b5-40be-ae94-8e23e0836d21&imsi_history=463&query=音乐&qn-sig=5bfecb6e5758e1f6b3ff9acb85da842a&activefrom=icon&imsi=463&omgid=cf268fc07aedcc46c33a7b078a18a2b9cd36b'|jq '.'|grep '\'url\':'|awk -F '\'' '{print $4}'

最后，得到URL，那么采集就比较简单了。可以利用火车头采集器或python脚本都可以。有什么问题可以咨询。