Unable to install Satpy on Raspberry Pi

[gersmit]

I'm Ger from the Netherlands, and I'm wondering if someone can explain what I'm doing wrong and why I can't install Satpy on a Raspberry Pi. Operating System: Raspbian GNU/Linux 10 (buster), Kernel: Linux 5.10.103-v7l+, Architecture: ARM.

I've been trying for two days and keep getting messages like: Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-g6om7379/pyproj/, Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-8koadk44/netCDF4/, and more similar errors. I ran pip3 install satpy, "satpy[all], ==0.19.0, -i https://www.piwheels.org/simple.
Nothing works, and it doesn't seem that difficult... yet. I hope someone has a solution for me, perhaps in a different way than the standard pip routine.

Thanks in advance for your help and best regards.

[djhoese]

Thanks for filing an issue. This should be interesting to figure out.

First, why are you forcing the version to 0.19.0? The latest version on PyPI is 0.58.0.

Second, the error seems to come from installing the netcdf4 library which is one possible dependency of satpy.

Do you know what Satpy readers and writers you want to use? If so, that would let you limit what dependencies are needed and save you a lot of headache on installation. Doing the "all" extra like you are includes all reader dependencies, writer dependencies, and everything in between. It's a lot.

[gersmit]

Hi David, nice to hear from you. I also reported the problem on November 22, 2021, downloading and installing Satpy for Raspberry Pi #1893.

I've had a crash and forgot to make a backup.
I've tried all the versions, but to no avail, so I think I'll use the version that matches the year of the Raspberry Pi image and that worked back then. Not now.

I believe nedcdf-4 can cause problems, but I also often get messages when installing pyproj and proj, and multiple installs.

I mainly use reader = "seviri_l1b_native" and the simple PNG writers and matplotlib.

David, is there a way to download a Raspberry Pi image with Satpy already installed?
I find it strange that no one else with Raspberry pi has this problem.
Okay, I'm not the youngest anymore, and amateur programming is my hobby and installing a program with PIP should be possible. Perhaps you can tell me how I can install the programs I need?

[djhoese]

Sorry, I have very little experience with raspberry pis. If you are only using that seviri reader and PNG saving then you definitely don't want to do satpy[all]. There is one dependency defined for that reader in Satpy's pyproject.toml file, but it is actually also a hard requirement of Satpy already (pyorbital) so doing pip install satpy should get you everything you need except for matplotlib.

[gersmit]

Okay, Matplotlib isn't included, I know that. But is there a way I can install Satpy with separate components?
Btw, you're talking about pyproject.toml. I keep getting error messages about that, and the installation stops there. How can I override that?
What do you recommend? A different OS? Or are there other installation options... I read something about Pytroll/Satpy?

[djhoese]

What do you mean separate components? The way Satpy is setup if you don't use a specific piece (ex. abi_l1b reader) then those dependencies aren't imported/needed (ex. netcdf4). You shouldn't use the "[all]" in your install command that you showed before.

Pyproject.toml is the modern way to describe a python package. 99% of modern/maintained packages will use that instead of or in addition to a setup.py file which you may be more familiar with. The error message will be for a specific dependency as it is being installed most likely. Some depend on C/C++ libraries to be compile-able on your system or pre-installed in the system directories. For example, pyproj will install the PROJ library if it isn't already available.

I can't make any reasonable recommendations unless you tell me what is wrong with your current installation method (where you don't install all of satpy's extra dependencies).

Pytroll is the community that maintains the satpy package. That's this repository. I'm not sure what you meant by that last question.

[djhoese]

Can you show me all the python package installation commands you're running?

[gersmit]

I hope the following is clear. These are common errors, but there are others as well. I keep having to load a different component with a different version?, but Satpy overwrites it??

What do you mean separate components? --> Years ago, I installed all the parts individually, one by one, and that worked. But I no longer have the list.
If you don't know a solution, I'll try with Linux or Ubuntu.... What do you think about this?

I'm using for install:
pip install satpy --extra-index-url https://www.piwheels.org/simple
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-i5zpZ4/zarr/

pip3 install satpy --extra-index-url https://www.piwheels.org/simple
No failure, short list, list completes but cannot find satpy

pip install satpy -i https://www.piwheels.org/simple
Collecting pyresample>=1.11.0 (from satpy), Could not find a version that satisfies the requirement pyresample>=1.11.0 (from satpy) (from versions: )
No matching distribution found for pyresample>=1.11.0 (from satpy)

pip3 install satpy -i https://www.piwheels.org/simple
None Error message: Short list completes list correctly but cannot find satpy

pip install satpy
Loading appears to be working fine. Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-zeFK3X/zarr/
Collecting zarr (from satpy)
Using cached https://files.pythonhosted.org/packages/d0/d0/02a3457f4e86a8f74104e7686c43b6bf0c76534b626f174eb20786a6f781/zarr-2.3.2.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "", line 1, in
File "/tmp/pip-install-zeFK3X/zarr/setup.py", line 60, in
license='MIT',
File "/usr/lib/python2.7/dist-packages/setuptools/init.py", line 144, in setup
_install_setup_requires(attrs)
File "/usr/lib/python2.7/dist-packages/setuptools/init.py", line 139, in _install_setup_requires
dist.fetch_build_eggs(dist.setup_requires)
File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 724, in fetch_build_eggs
replace_conflicting=True,
File "/usr/lib/python2.7/dist-packages/pkg_resources/init.py", line 782, in resolve
replace_conflicting=replace_conflicting
File "/usr/lib/python2.7/dist-packages/pkg_resources/init.py", line 1065, in best_match
return self.obtain(req, installer)
File "/usr/lib/python2.7/dist-packages/pkg_resources/init.py", line 1077, in obtain
return installer(requirement)
File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 791, in fetch_build_egg
return cmd.easy_install(req)
File "/usr/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 704, in easy_install
return self.install_item(spec, dist.location, tmpdir, deps)
File "/usr/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 730, in install_item
dists = self.install_eggs(spec, download, tmpdir)
File "/usr/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 901, in install_eggs
os.path.abspath(dist_filename)
distutils.errors.DistutilsError: Couldn't find a setup script in /tmp/easy_install-JijJar/setuptools_scm-9.2.1.tar.gz

----------------------------------------

Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-zeFK3X/zarr/

pip3 install satpy
If I think .... now installed properly?, run satpy, from satpy import Scene, error : File "pandas/_libs/interval.pyx", line 1, in init pandas._libs.interval
ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 44 from C header, got 40 from PyObject

[djhoese]

I hope the following is clear. These are common errors, but there are others as well. I keep having to load a different component with a different version?, but Satpy overwrites it??

What do you mean separate components? --> Years ago, I installed all the parts individually, one by one, and that worked.

What is a component?

If you don't know a solution, I'll try with Linux or Ubuntu.... What do you think about this?

Aren't you using linux now?

From your error messages above it is clear that your system has Python 2.7 installed and is using that. I forgot about this as a possibility so you should use pip3 or python3 -m pip for all your pip commands. You won't get far trying to install modern versions of packages with Python 2.

No failure, short list, list completes but cannot find satpy

What does "short list" refer to? And if it can't find Satpy then that is the error message, right? Posting the full output of something like pip3 install satpy would be helpful like you did above for the pip python-2 call above.

If you are using debian 10 (buster) then just based on https://www.debian.org/releases/ you should update your version. Maybe that's a different story for raspberry pis. Otherwise I don't think this is something that requires drastic changes in how you're doing stuff, but it is really hard to tell as it feels like you're only giving me half the information or my lack of rpi experience is causing me to make bad suggestions for you (I don't think that's true...yet).

[gersmit]

David, I'll send you some more, but you can enjoy your Sunday. The rest will come next week if you still have time.
I'm using Rasbian == Linux too, right?

Individual components are the packages that make up satpy.

"Short list" means a short summary of what is/is being loaded.

I've now run python3 -m pip satpy and I keep getting the message:
Collecting pyproj (from satpy)
Downloading pyproj-3.2.1.tar.gz (213 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 213.3/213.3 kB 1.5 MB/s eta 0:00:00
Installing build dependencies ... done
Getting requirements to build wheel ... error
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [3 lines of output]
/tmp/pip-install-btifnlms/pyproj_1965efbb231444c592a736df843165c8/setup.py:9: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
from pkg_resources import parse_version
proj executable not found. Please set the PROJ_DIR variable. For more information see: https://pyproj4.github.io/pyproj/stable/installation.html
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

[djhoese]

You may need to upgrade your version of raspberry pi os. What version of Python does the system have (python3 -V I think)? That version of pyproj it is trying to install is still 4 years old. You may need to install the PROJ library (libproj might also be the name) to your system. You could also install a separate version of python outside of system directories to give you more control over what versions are installed.

Bottom line is that these dependencies have C and C++ dependencies or other parts that need to be compiled. Normally they pre-build these for most platforms but not for older versions of python or older versions of linux.

Side note: I'm surprised an rpi has enough memory to do satellite data processing.

[gersmit]

I'll give it a try with your suggestions and see if it works. I'll check if there's a later OS version.
I'm currently using Python 3.7.3.

And yes, creating weather graphics is a breeze, as long as they're not too big. As a retiree, it's a nice hobby for me between other things.
However, I'll try Ubuntu next week; it might work better.

If you happen to learn anything more about my questions along the way, please let me know.

Thank you so much for listening and helping me.

Best regards, and maybe till later,
Ger