Thingspeak now unable to parse website

1 回表示 (過去 30 日間)
Mark Quinn
Mark Quinn 2024 年 10 月 31 日
コメント済み: Mark Quinn 2024 年 11 月 1 日
I have a ThingHTTP request that parses my local council website and extracts the bins which are collected each week. I then display this on an arduino with colour LEDs.
This worked fine for years but now for some reason the Thingspeak request cannot parse the website any more???
The conucil website is:
This isn't my address but a random house nearby ;)
I've copied the XPath for the string about this week (//*[@id="standard-page"]/div/div/div[2]/p[1]/text()[2])
Then used the GET request on thingspeak - This used to work fine.
Now it gives me :
Error parsing document, try a different parse string.
Any ideas?
  2 件のコメント
Steven Lord
Steven Lord 2024 年 11 月 1 日
You have confirmed that the layout of the page hasn't changed, and that the council hasn't changed who/what is allowed to access the website (choosing to use a robots.txt to block the scraping mechanism that ThingSpeak uses to read the data, for example?)
Mark Quinn
Mark Quinn 2024 年 11 月 1 日
The layout hasn't changed. I checked the XPath and I'm sure that's correct.
I don't know how to check if the website has a scraper blocker though - how would I go about checking for that?

サインインしてコメントする。

回答 (1 件)

Vinod
Vinod 2024 年 11 月 1 日
I can't access the URL from the public internet in the US. I believe this to be the reason the ThingSpeak servers can't get to the page and parse the content. I also tried to use Opera browser's VPN to see if this page is accessible in other locations, and it is not accessible.
Have there been changes to the page's availability outside the UK?
  3 件のコメント
Vinod
Vinod 2024 年 11 月 1 日
The one thought I had was to see if we could run it through Google Translate. However, it appears even Google Translate cannot get to the site. See:
The alternative I can think of is to find a free online proxy server based in the UK that can allow the page to be accessed outside the UK.
Mark Quinn
Mark Quinn 2024 年 11 月 1 日
That's a clever approach- pity it didn't work 😞
I think I might have to look at using the Arduino to parse the website directly rather than using an API... I can't imagine this being fun 😅

サインインしてコメントする。

コミュニティ

その他の回答  ThingSpeak コミュニティ

カテゴリ

Help Center および File ExchangeString Parsing についてさらに検索

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by