(不定期更新)使用VBA解決 excel web 查詢無法匯入、匯入太慢的股市資料

Dear Snare 大,

999程式可用了,我偷懶,先拿來用。
1000與 1024 繼續了解囉。

謝謝您的幫忙了。
snare大神, 你好:
請問想從Yahoo股市抓"類股報價"該怎麼抓?
對於類股股數比較少的, 已可用CreateObject("msxml2.xmlhttp")抓取, 但對於股票數比多的類股就只能抓到不完整的資料!
例如, 這"上市紡織分類行情", 剛進入這網頁時, 只會出現到"南緯1467.TW", 要再等一下才會出現完整的資料!
但就算是等完整資料都出現, 此時去看網頁原始碼, 還是只能看到至"南緯1467.TW", "南緯1467.TW"以下的都在網頁原始碼看不到! 這種情況下, VBA有辦法解嗎? 謝謝!
PS: 如果之前已回過這類似問題, 請恕小弟眼拙!

yth0315 wrote:
PS: 如果之前已回過這類似問題


1024樓

yth0315 wrote:
例如, 這"上市紡織分類行情", 剛進入這網頁時, 只會出現到"南緯1467.TW", 要再等一下才會出現完整的資料!
但就算是等完整資料都出現, 此時去看網頁原始碼, 還是只能看到至"南緯1467.TW", "南緯1467.TW"以下的都在網頁原始碼看不到! 這種情況下, VBA有辦法解嗎?


從網頁原始碼、f12開發者工具追踨網址可知,每次下載資料量只有30筆
超過30筆以上,網頁會載入2次,60筆以上3次……格式為json
所以原始碼中才會看不到資料



Sub Get_Yahoo_類股報價_Json()

Dim Xmlhttp As Object, Jsondata As Object, DecodeJson, Url As String, sectorId As String, exchange As String, p As Integer, Page As Integer, temp, i As Integer, ttt As Double

Sheets("工作表1").Cells.Clear
Sheets("工作表1").Range("a1:k1") = Array("股票名稱", "代號", "股價", "漲跌", "漲跌幅(%)", "開盤", "昨收", "最高", "最低", "成交量 (張)", "時間")
Application.ScreenUpdating = False

'sectorld、exchange,2個參數,同開啟網頁時,網址上的參數
'test
sectorId = "4"
exchange = "TAI"

'test
'sectorId = "33"
'exchange = "TAI"

'test
'sectorId = "123"
'exchange = "TWO"

'test
'sectorId = "167"
'exchange = "TWO"

'test
'sectorId = "166"
'exchange = "TWO"


ttt = Timer

Page = 1
p = 0


Do

Set Jsondata = CreateObject("HtmlFile")
Set Xmlhttp = CreateObject("WinHttp.WinHttpRequest.5.1")

'因語法關係無法正常顯示,Jsondata.write 這行程式碼,請手動輸入




Url = "https://tw.stock.yahoo.com/_td-stock/api/resource/StockServices.getClassQuotes;exchange=" & exchange & ";offset=" & p * 30 & ";sectorId=" & sectorId & "?bkt=tw-qsp-exp-no2-1&device=desktop&ecma=default&feature=ecmaModern&intl=tw⟪=zh-Hant-TW&partner=none®ion=TW&site=finance&tz=Asia/Taipei&returnMeta=true"

With Xmlhttp

.Open "GET", Url, False
.setRequestHeader "Content-Type", "application/x-www-form-urlencoded"
.setRequestHeader "Cache-Control", "no-cache"
.setRequestHeader "Pragma", "no-cache"
.setRequestHeader "If-Modified-Since", "Sat, 1 Jan 2000 00:00:00 GMT"
.send

End With


Set DecodeJson = CallByName(Jsondata.JsonParse(Xmlhttp.responsetext), "data", VbGet)

If p = 0 Then

Page = CallByName(CallByName(DecodeJson, "pagination", VbGet), "resultsTotal", VbGet)
Debug.Print Page 'total
If Page = 0 Then
MsgBox "暫無資料", vbOKOnly, "Error"
Application.ScreenUpdating = True
Exit Sub
End If

Page = WorksheetFunction.RoundUp(Page / 30, 0)

End If

With Sheets("工作表1")


For i = 0 To CallByName(CallByName(DecodeJson, "list", VbGet), "length", VbGet) - 1
Set temp = CallByName(CallByName(DecodeJson, "list", VbGet), i, VbGet)

.Cells(i + 2 + (p * 30), 1) = temp.symbolName
.Cells(i + 2 + (p * 30), 2) = CallByName(temp, "symbol", VbGet)
.Cells(i + 2 + (p * 30), 3) = CallByName(temp, "price", VbGet)
.Cells(i + 2 + (p * 30), 4) = CallByName(temp, "change", VbGet)
.Cells(i + 2 + (p * 30), 5) = "'" & temp.changePercent
.Cells(i + 2 + (p * 30), 6) = temp.regularMarketOpen
.Cells(i + 2 + (p * 30), 7) = temp.regularMarketPreviousClose
.Cells(i + 2 + (p * 30), 8) = temp.regularMarketDayHigh
.Cells(i + 2 + (p * 30), 9) = temp.regularMarketDayLow
.Cells(i + 2 + (p * 30), 10) = temp.volumeK
.Cells(i + 2 + (p * 30), 11) = temp.regularMarketTime

Next i


End With


Set Xmlhttp = Nothing
Set Jsondata = Nothing
Set DecodeJson = Nothing
Set temp = Nothing

p = p + 1

'Delaytick(0.5)

Loop Until p = Page


Sheets("工作表1").Cells.EntireColumn.AutoFit
Application.ScreenUpdating = True
Debug.Print Timer - ttt & "s"



End Sub


Sub Delaytick(setdelay As Single)

Dim StartTime As Double, NowTime As Double
StartTime = Timer * 100
setdelay = setdelay * 100
Do
NowTime = Timer * 100
DoEvents
Loop Until NowTime - StartTime > setdelay

End Sub



snare大神, 你好:
大神不愧是大神, 這麼短的時間內就能將小弟長期的問題解決, 感激不盡!
不過, 小弟可能還是無福消受! 當程式執行時, 發生物件不支援的問題! 應該是因為我用的Excel版本是2002, 這阿公級的版本無法run Jsondata.JsonParse.
可能小弟要把Xmlhttp.responsetext的內容好好study之後才能知道有沒有辦法避掉這問題吧?
總之, 再次感謝snare大神!
yth0315 wrote:
當程式執行時, 發生物件不支援的問題! 應該是因為我用的Excel版本是2002, 這阿公級的版本無法run Jsondata.JsonParse.
可能小弟要把Xmlhttp.responsetext的內容好好study之後才能知道有沒有辦法避掉這問題吧?


就是為了相容性,我才用CreateObject("HtmlFile"),建立一個Jsondata.JsonParse副程式
如果您曾經看過這幾年任何一個json範例的文章
就會發現您不能用,是因為程式碼漏了一行
我重新加上註解了,請回頭重看文章,不要直接複製=>貼上
或從以前有上傳檔案的範例中,直接複製程式碼


我沒有excel 2002,但2000(xp)、2003(xp)是正常的



yth0315
snare大神, 你好:果真加了那行"副程式"後就可執行無誤!讓小弟對您的景仰有如滔滔江水, 綿延不絕! 也請恕小弟稍早前的疏忽!
請問版主:vba加密,有那些方法?謝謝你.
snare
方法請google 關鍵字vba加密、隱藏巨集、隱藏專案…但只要是excel內建的功能,基本上都沒用,網路上一堆破解教學,有心人士幾分鐘就破解了,excel轉exe、或做成增益集,保護力比較夠
Hi snare大大
https://tw.stock.yahoo.com/q/bc?s=2330
之前參考你的程式用來收集每日的收盤價,
這兩天不知道是不是因為yahoo改版.看網頁內容跟網址跟之前不一樣了.
嘗試更改網址及table都抓不到了.
請snare大大指指導要怎麼做修改.
謝謝.
yingchieh
謝謝snare大大,幾乎換sheet名稱就可以用.我大約抓700多檔,發現中間會有下載fail外,大約約600檔後全fail.是因為網頁改版的原因嗎?會擋下載太多次?
snare
測試後發現大量下載時,確實容易出現下載失敗(可自行增加廻圈重新下載),3個下載方式中,新版、舊版、json,猜測json可能不會有這個問題,但還沒空測試,如果只是收盤後使用,也可先用269樓範例代替
Snare大您好:
vba上星期五還能用,這星期一就出現 "automation 錯誤"在"Set DecodeJson = Jsondata.JsonParse(.Responsetext)"

vba如下
Sub mainbsn()

Dim i As Integer

With Sheets("bsn")

If .Cells(2, 1) = "" Then Exit Sub
'Sheets("bsn").Cells.Range("d2:h100").ClearContents
.Select
'.Range(.Cells(2, 2), .Cells(.UsedRange.Rows.Count, .UsedRange.Columns.Count)).clear
'.Range("A1:L1") = Array("股票代號", "股票名稱", "時間", "成交", "買進", "賣出", "漲跌", "張數", "昨收", "開盤", "最高", "最低")
.Columns("A:A").NumberFormatLocal = "@"
.Columns.AutoFit
.Cells.HorizontalAlignment = xlRight
.Range("A:A,B:B,C:C,G:G").HorizontalAlignment = xlLeft
.Range("A:A").ColumnWidth = 10
.Range("B:B").ColumnWidth = 13
.Range("C:C,D:D,E:E,F:F,G:G,H:H,I:I,J:J,K:K,L:L").ColumnWidth = 8

For i = 2 To .Range("a1").CurrentRegion.Rows.Count
DoEvents
Call bsn(.Cells(i, 1), i, True, 1)
If i = Sheets("capital").Range("b1") - 1 Then Exit Sub
Next i

.Cells(1, 1).Select

End With


End Sub



Sub bsn(stock_id As String, lastrow As Integer, sync As Boolean, timeout As Long)


Dim Xmlhttp As Object, Jsondata As Object, DecodeJson, Url As String, urla As String, ttt As Double

Set Jsondata = CreateObject("HtmlFile")
Set Xmlhttp = CreateObject("Msxml2.XMLHTTP")
Jsondata.Write ""


'Sheets("bsn").Cells.Range("c2:aa100").ClearContents
Sheets("bsn").Range("c1:h1") = Array("date", "close", "stockAgentMainPower", "stockAgentDiff", "skp5", "skp20")
ttt = Timer

Url = "https://www.wantgoo.com/stock/" & stock_id & "/major-investors/main-trend-data"
urla = "https://www.wantgoo.com/stock/" & stock_id & "/major-investors/main-trend"

Set Xmlhttp = CreateObject("msxml2.xmlhttp")
With Xmlhttp
.Open "GET", Url, False
.setRequestHeader "Content-Type", "application/x-www-form-urlencoded"
.setRequestHeader "Referer", urla
.send
Set DecodeJson = Jsondata.JsonParse(.Responsetext)
End With

With Sheets("bsn")
Application.ScreenUpdating = False
For i = 0 To 0
Dim dt As String: dt = Left$(CallByName(CallByName(DecodeJson, i, VbGet), "date", VbGet), 13)
Dim off As String: off = Mid$(CallByName(CallByName(DecodeJson, i, VbGet), "date", VbGet), 14)
Dim d As Date: d = DateAdd("s", CCur(dt) / 1000, "01/02/1970")
'd = DateAdd("h", Left$(off, 3), d)
'd = DateAdd("n", Right$(off, 2), d)
'd1 = d.ToString("yyyy/MM/dd")

'.Cells(i + 2, 1) = Left(CallByName(CallByName(DecodeJson, i, VbGet), "date", VbGet), 10)
.Cells(i + lastrow, 9) = DateValue(d)
'.Cells(i + lastrow, 3) = Left(CallByName(CallByName(DecodeJson, i, VbGet), "date", VbGet), 10)
.Cells(i + lastrow, 4) = CallByName(CallByName(DecodeJson, i, VbGet), "close", VbGet)
.Cells(i + lastrow, 5) = CallByName(CallByName(DecodeJson, i, VbGet), "stockAgentMainPower", VbGet)
.Cells(i + lastrow, 6) = CallByName(CallByName(DecodeJson, i, VbGet), "stockAgentDiff", VbGet)
.Cells(i + lastrow, 7) = CallByName(CallByName(DecodeJson, i, VbGet), "skp5", VbGet)
.Cells(i + lastrow, 8) = CallByName(CallByName(DecodeJson, i, VbGet), "skp20", VbGet)
'.Cells(i + lastrow, 9) =
Next i
.Cells.EntireColumn.AutoFit
Application.ScreenUpdating = True
End With

' MsgBox "(" & Left(CallByName(CallByName(DecodeJson, i - 1, VbGet), "date", VbGet), 10) & _
")~(" & Left(CallByName(CallByName(DecodeJson, 0, VbGet), "date", VbGet), 10) & ")" & _
vbNewLine & CallByName(DecodeJson, "length", VbGet) & "筆" & vbNewLine & Timer - ttt & "秒"

Set Xmlhttp = Nothing
Set DecodeJson = Nothing


End sub
找了一天,看了前面的文章,實在找不到,請Snare大指點一下,謝謝您!
snare
2021/07/28 01:45 用您的程式碼測試,可正常跑完,但有些小錯誤,稍微修正一下,請參考1059樓




'執行test
Sub test()

Dim lastrow As Integer, i As Integer, ttt As Double

ttt = Timer
Cells.Clear
Sheets("工作表1").Range("a1:g1") = Array("stock", "date", "close", "stockAgentMainPower", "stockAgentDiff", "skp5", "skp20")

Application.ScreenUpdating = False

For i = 1 To 5 '測試5筆
Call WantGoo_Json_test(Choose(i, "2330", "2412", "2002", "2603", "2303"), i + 1)
Next i
Sheets("工作表1").Cells.EntireColumn.AutoFit
Application.ScreenUpdating = True
Debug.Print Timer - ttt & "s"

End Sub


Sub WantGoo_Json_test(stock_id As String, lastrow As Integer)

Dim Xmlhttp As Object, Jsondata As Object, DecodeJson, Url As String, urla As String
Set Jsondata = CreateObject("HtmlFile")
Set Xmlhttp = CreateObject("Msxml2.XMLHTTP")






Url = "https://www.wantgoo.com/stock/" & stock_id & "/major-investors/main-trend-data"
urla = "https://www.wantgoo.com/stock/" & stock_id & "/major-investors/main-trend"


With Xmlhttp
.Open "GET", Url, False
.setRequestHeader "Content-Type", "application/json; charset=utf-8"
.setRequestHeader "Referer", urla
.setRequestHeader "User-Agent", "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.107 Safari/537.36"
.send
Set DecodeJson = Jsondata.JsonParse(.Responsetext)
End With

With Sheets("工作表1")
.Cells(lastrow, 1) = stock_id
.Cells(lastrow, 2) = Format(CallByName(CallByName(DecodeJson, 1, VbGet), "date", VbGet) / 86400000 + #1/1/1970 8:00:00 AM#, "yyyy/mm/dd")
.Cells(lastrow, 3) = CallByName(CallByName(DecodeJson, 0, VbGet), "close", VbGet)
.Cells(lastrow, 4) = CallByName(CallByName(DecodeJson, 0, VbGet), "stockAgentMainPower", VbGet)
.Cells(lastrow, 5) = CallByName(CallByName(DecodeJson, 0, VbGet), "stockAgentDiff", VbGet)
.Cells(lastrow, 6) = CallByName(CallByName(DecodeJson, 0, VbGet), "skp5", VbGet)
.Cells(lastrow, 7) = CallByName(CallByName(DecodeJson, 0, VbGet), "skp20", VbGet)
End With


Set Xmlhttp = Nothing
Set DecodeJson = Nothing


End Sub



goldchiou
Snare大,謝謝您!
Snare大您好:
想從
https://mops.twse.com.tw/mops/web/t116sb01_new
上市中的 [股款或價款繳納完成日起十五日內應申報相關資訊] 抓資料(個股名稱 股號 私募到期日 私募金額)。
您在965樓有類似範例 但在我win7 及 excel2013(64bits) 抓無資料 ,不知我哪裡沒設定好? 能否請Snare大抽空指導要抓上述資料要如何下手 謝謝!
snare
請參考1062樓範例,寫法類似965樓,網頁連結下載資料有需要請自行修改Get_twse_詳細資料()副程式,下次有問題請附檔案或程式碼,我才能知道您弄錯什麼地方
文章分享
評分
評分
複製連結
請輸入您要前往的頁數(1 ~ 157)

今日熱門文章 網友點擊推薦!