Orbital cannot get TLEs from the internet

[anikfal]

It's been a couple of days that Orbital has stopped functioning for seemingly all TLEs (same result for Meteosat-8, Suomi NPP, etc.):

from pyorbital.orbital import Orbital
orb = Orbital("NOAA-20")

Output:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/sebal/miniconda3/envs/mysat/lib/python3.9/site-packages/pyorbital/orbital.py", line 166, in __init__
    self.tle = tlefile.read(satellite, tle_file=tle_file,
  File "/home/sebal/miniconda3/envs/mysat/lib/python3.9/site-packages/pyorbital/tlefile.py", line 106, in read
    return Tle(platform, tle_file=tle_file, line1=line1, line2=line2)
  File "/home/sebal/miniconda3/envs/mysat/lib/python3.9/site-packages/pyorbital/tlefile.py", line 154, in __init__
    self._read_tle()
  File "/home/sebal/miniconda3/envs/mysat/lib/python3.9/site-packages/pyorbital/tlefile.py", line 212, in _read_tle
    fid = open_func(url)
  File "/home/sebal/miniconda3/envs/mysat/lib/python3.9/urllib/request.py", line 214, in urlopen
    return opener.open(url, data, timeout)
  File "/home/sebal/miniconda3/envs/mysat/lib/python3.9/urllib/request.py", line 523, in open
    response = meth(req, response)
  File "/home/sebal/miniconda3/envs/mysat/lib/python3.9/urllib/request.py", line 632, in http_response
    response = self.parent.error(
  File "/home/sebal/miniconda3/envs/mysat/lib/python3.9/urllib/request.py", line 555, in error
    result = self._call_chain(*args)
  File "/home/sebal/miniconda3/envs/mysat/lib/python3.9/urllib/request.py", line 494, in _call_chain
    result = func(*args)
  File "/home/sebal/miniconda3/envs/mysat/lib/python3.9/urllib/request.py", line 747, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
  File "/home/sebal/miniconda3/envs/mysat/lib/python3.9/urllib/request.py", line 523, in open
    response = meth(req, response)
  File "/home/sebal/miniconda3/envs/mysat/lib/python3.9/urllib/request.py", line 632, in http_response
    response = self.parent.error(
  File "/home/sebal/miniconda3/envs/mysat/lib/python3.9/urllib/request.py", line 561, in error
    return self._call_chain(*args)
  File "/home/sebal/miniconda3/envs/mysat/lib/python3.9/urllib/request.py", line 494, in _call_chain
    result = func(*args)
  File "/home/sebal/miniconda3/envs/mysat/lib/python3.9/urllib/request.py", line 641, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found

pyorbital version: 1.6.1
OS: Centos 7

[dagnic]

See: #138. In the meantime you could use my fix: https://github.com/dagnic/pyorbital/tree/main, just clone and pip install.

[anikfal]

That is interesting.
Today around 13:00 UTC, I came across to the celestrak new URLs. However, the python requests could not get the new URLs. So modifying pyorbital/tlefile.py and replacing TLE_URLS with the new URLs was not effective for my case.

import requests
url = `https://celestrak.org/NORAD/elements/gp.php?GROUP=noaa&FORMAT=tle`
requests.get(url)

output:
<Response [403]>

---

Anyway, it is now working, and the output of requests.get(url) is Response [200]. I don't know what the problem was just a couple of hours ago.

[anikfal]

<Response [403]>

This problem was due to a download limit (excessive use). After 2 hours the block is removed.

[dagnic]

I don't think the issue should be closed as the PR for the original issue is not approved yet.

The second issue you described is well explained in this post by T.S. Kelso.

Automated scripts should check the age of the user’s local data and not re-download unless it is more than 2 hours old.

This behaviour is not well supported by pyorbital and, maybe, we should consider to implement it: download once, save in TLES path, redownload only if file doesn't exist or is older than 2 hours.

[lahorite]

from pyorbital import tlefile
tle = tlefile.read('NOAA-21')
tle.orbit
num_of_orbits = tle.orbit

I am getting this error -

HTTPError Traceback (most recent call last)
~\AppData\Local\Temp\ipykernel_40736\3510404166.py in
1 from pyorbital import tlefile
----> 2 tle = tlefile.read('NOAA-21')
3 tle.orbit
4 num_of_orbits = tle.orbit

~\Anaconda3\lib\site-packages\pyorbital\tlefile.py in read(platform, tle_file, line1, line2)
158
159 """
--> 160 return Tle(platform, tle_file=tle_file, line1=line1, line2=line2)
161
162

~\Anaconda3\lib\site-packages\pyorbital\tlefile.py in init(self, platform, tle_file, line1, line2)
204 self.orbit = None
205
--> 206 self._read_tle()
207 self._checksum()
208 self._parse_tle()

~\Anaconda3\lib\site-packages\pyorbital\tlefile.py in _read_tle(self)
242 else:
243 uris, open_func = _get_uris_and_open_func(tle_file=self._tle_file)
--> 244 tle = _get_first_tle(uris, open_func, platform=self._platform)
245
246 if not tle:

~\Anaconda3\lib\site-packages\pyorbital\tlefile.py in _get_first_tle(uris, open_func, platform)
337
338 def _get_first_tle(uris, open_func, platform=''):
--> 339 return _get_tles_from_uris(uris, open_func, platform=platform, only_first=True)
340
341

~\Anaconda3\lib\site-packages\pyorbital\tlefile.py in _get_tles_from_uris(uris, open_func, platform, only_first)
344 designator = "1 " + SATELLITES.get(platform, '')
345 for url in uris:
--> 346 fid = open_func(url)
347 for l_0 in fid:
348 tle = ""

~\Anaconda3\lib\urllib\request.py in urlopen(url, data, timeout, cafile, capath, cadefault, context)
212 else:
213 opener = _opener
--> 214 return opener.open(url, data, timeout)
215
216 def install_opener(opener):

~\Anaconda3\lib\urllib\request.py in open(self, fullurl, data, timeout)
521 for processor in self.process_response.get(protocol, []):
522 meth = getattr(processor, meth_name)
--> 523 response = meth(req, response)
524
525 return response

~\Anaconda3\lib\urllib\request.py in http_response(self, request, response)
630 # request was successfully received, understood, and accepted.
631 if not (200 <= code < 300):
--> 632 response = self.parent.error(
633 'http', request, response, code, msg, hdrs)
634

~\Anaconda3\lib\urllib\request.py in error(self, proto, *args)
559 if http_err:
560 args = (dict, 'default', 'http_error_default') + orig_args
--> 561 return self._call_chain(*args)
562
563 # XXX probably also want an abstract factory that knows when it makes

~\Anaconda3\lib\urllib\request.py in _call_chain(self, chain, kind, meth_name, *args)
492 for handler in handlers:
493 func = getattr(handler, meth_name)
--> 494 result = func(*args)
495 if result is not None:
496 return result

~\Anaconda3\lib\urllib\request.py in http_error_default(self, req, fp, code, msg, hdrs)
639 class HTTPDefaultErrorHandler(BaseHandler):
640 def http_error_default(self, req, fp, code, msg, hdrs):
--> 641 raise HTTPError(req.full_url, code, msg, hdrs, fp)
642
643 class HTTPRedirectHandler(BaseHandler):

HTTPError: HTTP Error 404: Not Found

[djhoese]

@lahorite Are you pointing out something new about this issue? Otherwise your issue seems the same as what is already discussed and has a workaround in #138.

[lahorite]

Its the same - its fixed now - I copied the latest tlefile.py - really appreciate it @djhoese

[adybbroe]

Thanks for raising this and preparing a solution @dagnic
I add the direct link to the explanations by Dr. Kelso here for completeness:
https://celestrak.org/NORAD/documentation/gp-data-formats.php

[djhoese]

Closed via #138 and just released on PyPI as 1.8.1. A conda-forge package will come later today.