Release OzWeather - Australian Weather Addon using BOM data inc. animated radar support
(2021-04-28, 00:31)bossanova808 Wrote: IMPORTANT UPDATE:

There are lots of changes going on with the BOM data/services right now.  I am busy re-writing things to use the new BOM API for weather observation/forecast data.


But - also - after being able to download the radars just fine for ~10 years now, this morning instead of radar backgrounds, I see this message:

Potential automated request detected! We are making changes to our website therefore web scraping is no longer supported. Please contact us by filling in the details at http://reg.bom.gov.au/screenscraper/scre...uiry_form/ and we will get in touch with you.

Looks like I may have even more re-writing to do!  I had hoped just to release the new BOM API data part first (and am a fair way through that now) - without having to re-code the radar side of things.  But that is now looking less likely.  The message is odd as no scraping is involved in the radars, it's done with anonymous FTP as per the BOMs process for this.  But looks like they are blocking anonymous access.  I have contacted them and will see what they say, anyway...


Thus I suspect there is going to be a few weeks of breakage with OzWeather before things are working again (and I am presuming there will be public, free access to the radars again, somehow...). 
I suppose it is not surprising - 10 years is a long time to go without major changes, especially for something that scrapes, rather than uses a proper API.

So...the good news is weather data will soon use the BOM API (primarily because WeatherZone forecast data is no longer coming from the BOM it seems - and also because using an API is much better than scraping, anyway!).
The bad news is I have to find out what is going to happen with radars and that is much trickier coding so might take a while for me to work it all out. 

When the new release comes, everyone will have to re-configure the addon so that the addon knows the geohash for you location.  This will be built in, but it will mean you need to visit settings and update each of your locations to get things working again.

Yeah. The original Home Assistant BOM integration used web scraping for the observational data and it stopped working a few days ago as well. Fortunately Home Assistant had deprecated that a few months ago and another user wrote a new custom integration using the API as well but still some people didn't bother switching over to that and now it's just stopped working they have to reconfigure their systems...

We found this announcement:
```
Service Announcements
All scheduled release dates are subject to change due to technical and operational requirements.
Website notification of change
Scheduled Release Date: 3 March 2021A web application firewall policy has been implemented for www.bom.gov.au which will block screen scraping activity.The Bureau is monitoring screen scraping activity on the site and will commence interrupting, and eventually blocking, this activity on www.bom.gov.au from Wednesday, 3 March 2021. This is aimed at protecting infrastructure, system access and security, intellectual property and server/service load.Web or screen scraping is the act of copying information that shows on a digital display so it can be used for another purpose. This activity has always been at odds with the Bureau’s terms and conditions. We understand www.bom.gov.au contributes significantly to the work of many individuals and organisations and we are committed to continuing to provide access through our registered user’s channel.For further information, or to discuss the ongoing use of our materials, please make contact with us via [email protected].
```
Reply


Messages In This Thread
:grin::grin::grin: - by fastdump - 2011-12-15, 11:53
[No subject] - by bossanova808 - 2011-12-15, 11:55
[No subject] - by fastdump - 2011-12-15, 12:09
[No subject] - by bossanova808 - 2011-12-15, 12:10
[No subject] - by MaxHeadroom - 2011-12-15, 12:23
[No subject] - by N3MIS15 - 2011-12-15, 12:28
[No subject] - by bossanova808 - 2011-12-15, 12:43
[No subject] - by bossanova808 - 2011-12-16, 02:06
[No subject] - by fastdump - 2011-12-16, 04:30
[No subject] - by bossanova808 - 2011-12-16, 04:41
[No subject] - by bossanova808 - 2011-12-16, 05:16
[No subject] - by bossanova808 - 2011-12-16, 06:48
[No subject] - by rocketpants - 2011-12-16, 09:38
[No subject] - by fastdump - 2011-12-16, 11:30
[No subject] - by bossanova808 - 2011-12-17, 00:11
[No subject] - by fastdump - 2011-12-17, 04:40
[No subject] - by fastdump - 2011-12-17, 08:30
[No subject] - by bossanova808 - 2011-12-17, 08:32
[No subject] - by bossanova808 - 2011-12-17, 08:33
[No subject] - by fastdump - 2011-12-17, 11:00
[No subject] - by bossanova808 - 2011-12-17, 11:06
not working at all for me - by technophobe - 2011-12-19, 07:13
[No subject] - by bossanova808 - 2011-12-19, 07:20
[No subject] - by bossanova808 - 2011-12-19, 10:37
[No subject] - by bossanova808 - 2011-12-19, 12:25
log file? - by technophobe - 2011-12-20, 06:42
[No subject] - by bossanova808 - 2011-12-20, 11:47
[No subject] - by zan5hin - 2012-01-01, 13:50
[No subject] - by bossanova808 - 2012-01-03, 12:25
[No subject] - by bossanova808 - 2012-01-03, 13:25
[No subject] - by Troyp42 - 2012-01-11, 01:59
[No subject] - by bossanova808 - 2012-01-11, 02:01
[No subject] - by Troyp42 - 2012-01-11, 02:05
[No subject] - by bossanova808 - 2012-01-11, 02:07
[No subject] - by Troyp42 - 2012-01-11, 02:27
[No subject] - by bossanova808 - 2012-01-11, 11:51
[No subject] - by bossanova808 - 2012-01-11, 11:52
[No subject] - by Kokonutcreme - 2012-01-12, 07:52
[No subject] - by bossanova808 - 2012-01-12, 07:54
[No subject] - by Troyp42 - 2012-01-12, 08:30
[No subject] - by teeedubb - 2012-01-12, 09:14
[No subject] - by bossanova808 - 2012-01-12, 09:55
[No subject] - by teeedubb - 2012-01-12, 23:08
[No subject] - by fastdump - 2012-01-14, 18:02
[No subject] - by bossanova808 - 2012-01-15, 12:03
[No subject] - by bossanova808 - 2012-01-16, 13:17
[No subject] - by bossanova808 - 2012-01-16, 13:18
[No subject] - by bossanova808 - 2012-01-17, 02:30
[No subject] - by bossanova808 - 2012-01-17, 04:49
[No subject] - by Troyp42 - 2012-01-17, 09:31
[No subject] - by bossanova808 - 2012-01-17, 10:27
[No subject] - by bossanova808 - 2012-01-17, 11:45
[No subject] - by Troyp42 - 2012-01-17, 12:06
[No subject] - by bossanova808 - 2012-01-17, 12:09
[No subject] - by Troyp42 - 2012-01-17, 12:13
[No subject] - by bossanova808 - 2012-01-17, 12:16
[No subject] - by fastdump - 2012-01-17, 12:28
[No subject] - by bossanova808 - 2012-01-17, 12:32
[No subject] - by bossanova808 - 2012-01-17, 12:34
[No subject] - by Troyp42 - 2012-01-17, 12:39
[No subject] - by bossanova808 - 2012-01-17, 12:47
[No subject] - by fastdump - 2012-01-17, 12:57
[No subject] - by Troyp42 - 2012-01-17, 13:00
[No subject] - by bossanova808 - 2012-01-17, 13:03
[No subject] - by Troyp42 - 2012-01-17, 13:04
[No subject] - by bossanova808 - 2012-01-17, 13:06
[No subject] - by Troyp42 - 2012-01-17, 13:18
[No subject] - by bossanova808 - 2012-01-17, 13:26
[No subject] - by Troyp42 - 2012-01-17, 13:35
[No subject] - by bossanova808 - 2012-01-17, 13:42
[No subject] - by Troyp42 - 2012-01-17, 13:47
[No subject] - by bossanova808 - 2012-01-17, 13:51
[No subject] - by fastdump - 2012-01-17, 14:34
[No subject] - by bossanova808 - 2012-01-17, 14:39
[No subject] - by fastdump - 2012-01-17, 14:51
[No subject] - by bossanova808 - 2012-01-17, 14:55
[No subject] - by fastdump - 2012-01-17, 14:55
[No subject] - by bossanova808 - 2012-01-17, 14:56
[No subject] - by fastdump - 2012-01-17, 15:01
[No subject] - by bossanova808 - 2012-01-17, 15:08
[No subject] - by fastdump - 2012-01-17, 15:28
[No subject] - by bossanova808 - 2012-01-18, 01:09
[No subject] - by ya.rahman - 2012-01-18, 03:51
Dharma version - by ya.rahman - 2012-01-18, 03:53
[No subject] - by bossanova808 - 2012-01-18, 03:54
[No subject] - by ya.rahman - 2012-01-18, 04:27
[No subject] - by bossanova808 - 2012-01-18, 04:29
[No subject] - by bossanova808 - 2012-01-19, 14:41
[No subject] - by fastdump - 2012-01-20, 02:42
[No subject] - by bossanova808 - 2012-01-20, 05:41
[No subject] - by bossanova808 - 2012-01-20, 13:53
[No subject] - by bossanova808 - 2012-01-22, 07:01
[No subject] - by bossanova808 - 2012-01-22, 07:28
[No subject] - by blaher - 2012-01-25, 08:49
[No subject] - by bossanova808 - 2012-01-25, 11:18
[No subject] - by fastdump - 2012-01-26, 03:31
[No subject] - by blaher - 2012-01-26, 04:06
[No subject] - by bossanova808 - 2012-01-26, 04:55
[No subject] - by Troyp42 - 2012-01-29, 04:41
[No subject] - by bossanova808 - 2012-01-29, 05:19
[No subject] - by bossanova808 - 2012-01-29, 05:23
[No subject] - by bossanova808 - 2012-01-29, 06:03
[No subject] - by Troyp42 - 2012-01-29, 07:28
[No subject] - by Troyp42 - 2012-01-29, 13:07
[No subject] - by bossanova808 - 2012-01-30, 01:38
[No subject] - by Troyp42 - 2012-01-30, 03:33
[No subject] - by fastdump - 2012-01-30, 13:22
[No subject] - by Troyp42 - 2012-01-31, 08:20
[No subject] - by bossanova808 - 2012-01-31, 08:25
[No subject] - by Troyp42 - 2012-01-31, 23:48
[No subject] - by Freakish - 2012-02-01, 01:06
[No subject] - by bossanova808 - 2012-02-01, 01:10
[No subject] - by Freakish - 2012-02-01, 01:32
[No subject] - by bossanova808 - 2012-02-01, 01:36
[No subject] - by Freakish - 2012-02-01, 01:57
[No subject] - by bossanova808 - 2012-02-01, 02:02
[No subject] - by kozak069 - 2012-02-05, 02:37
[No subject] - by bossanova808 - 2012-02-05, 04:28
[No subject] - by kozak069 - 2012-02-05, 10:30
[No subject] - by bossanova808 - 2012-02-05, 10:37
[No subject] - by bossanova808 - 2012-02-08, 03:08
[No subject] - by bossanova808 - 2012-02-10, 06:19
[No subject] - by GiJOr33 - 2012-02-17, 06:39
Link not working? - by Manicman - 2012-02-26, 08:06
[No subject] - by scotia - 2012-02-27, 04:56
[No subject] - by bossanova808 - 2012-02-27, 05:14
[No subject] - by bossanova808 - 2012-02-27, 05:32
[No subject] - by bossanova808 - 2012-02-27, 05:33
[No subject] - by scotia - 2012-02-27, 05:48
[No subject] - by bossanova808 - 2012-02-27, 05:49
[No subject] - by bossanova808 - 2012-02-27, 07:18
[No subject] - by scotia - 2012-02-27, 07:19
[No subject] - by bossanova808 - 2012-02-27, 07:25
New code - by scotia - 2012-02-27, 07:29
[No subject] - by bossanova808 - 2012-02-27, 07:36
[No subject] - by bossanova808 - 2012-02-27, 07:43
[No subject] - by scotia - 2012-02-27, 07:49
[No subject] - by bossanova808 - 2012-02-27, 07:51
[No subject] - by scotia - 2012-02-27, 07:55
[No subject] - by bossanova808 - 2012-02-27, 08:00
OZWeather Error - by Jolly181 - 2012-03-11, 04:00
[No subject] - by bossanova808 - 2012-03-11, 05:01
[No subject] - by bossanova808 - 2012-03-11, 05:30
[No subject] - by Jolly181 - 2012-03-11, 06:08
[No subject] - by bossanova808 - 2012-03-11, 06:15
[No subject] - by bossanova808 - 2012-03-11, 06:15
[No subject] - by Jolly181 - 2012-03-11, 06:19
Neon compatible MyWeather.xml - by User 35611 - 2012-04-07, 14:51
peakay - by peakay - 2015-12-28, 14:06
peakay - by peakay - 2015-12-29, 02:45
RE: OzWeather - Australian Weather Addon using BOM data inc. animated radar support - by DavidFW1960 - 2021-04-28, 04:13
OzWeather not working - by awfma - 2023-09-06, 08:47
Logout Mark Read Team Forum Stats Members Help
OzWeather - Australian Weather Addon using BOM data inc. animated radar support5