Python 3.9 support

1

I understand that python 3.9 is not supported. Will l it be eventually and when ?

2

Hmm, I think @Alex is the best person to answer this

3

Hey, I think our position on python 3.9 support is that we would love to support it, but supporting it seems to be non trivial and without a compelling reason, it isn’t a high priority right now.

So I guess the answer is: it will eventually be supported, but we don’t know when yet.

4

@Robert_Charette what type of setup do you have? is 3.9 a blocker for you in using Ray?

5

Ray is used in a python program to perform the following tasks:

  1. read influenza hemagglutinin protein amino acid sequences from a database;
  2. from those sequences extract six subsets of amino acids (epitopes) that are specifically recognized by the immune system;
  3. within each subset compute the hamming distance between sequences;
  4. combine the six hamming distance to compute what we call an epitope distance matrix between all sequences ;
  5. compute multidimensional scaling (MDS) to visualize the epitope distances between influenza strains.

To increase speed I decide to go for parallel computing. Initially I used the multiprocessing library. No being a professional programmer I found it not to be easy and intuitive to use. I found ray to be easier to use. The improvement in computing a bliss.

I have two setups:

The first is my development machine running Manjaro Linux (kernel 5.4.85-1). It was running Python 3.8.6 until the last update when it turned to Python 3.9.1 (default, Dec 13 2020, 11:55:53) [GCC 10.2.0]. Pip refuses to install ray (ERROR: Could not find a version that satisfies the requirement ray, ERROR: No matching distribution found for ray) . I tried compiling from sources, that is when I noticed 3.9 was not supported. I tried downgrading back to 3.8.6 but that created other problems. I use Spyder as a Python development environment.

The second is a Debian GNU/Linux, kernel 4.19.0-13-amd64 running Python 3.7.3 (default, Jul 25 2020, 13:03:44) [GCC 8.3.0]. It runs the database, it is used by the people managing the sequences and for them to visualize MDS three dimensional plots. I have warned the system admin of that server not to upgrade python beyond 3.8.

The main issue for me is not being able anymore to develop further on my dev machine. The alternative could be to set up a virtual environment with a Python 3.8. I tried this for another issue but I found it very cumbersome to operate.

6

Can you try using Miniconda and creating a new environment with python=3.8?

7

I installed python 3.8.7 using pyenv. It does the job.

2 Likes
8

To add a note here. Our organization is appreciating the new features in 3.9 and of course 3.10 after some additional changes brought into core python with structural pattern matching, which is one of the interested features our codebases can leverage once its released. But since Ray doesn’t support anything > 3.8. We’re waiting for an official wheel to be announced instead of building our own. Any pointers on supporting higher python versions apart from 3.8 in the near future?

9

Just an update here: we’re now building experimental nightly wheels for 3.9. The usual nightly wheel installation instructions should “just work” for 3.9 now.

If all goes well, we should publish some PyPI wheels with Ray 1.5.

https://docs.ray.io/en/master/installation.html#daily-releases-nightlies

1 Like
10

Another update. Ray 1.4.1 is out, and we’ve published python 3.9 wheels for it. You should now be able to pip install ray on python 3.9 :slight_smile:

1 Like
11

@Alex Ray 1.10.0 docs still say “Python 3.9 support is currently experimental.”

Is that still true, or is it safe to move to 3.9 (or even 3.10)?

12

Thanks for thing ping, I’ll promote it to stable. It should be safe to move to 3.9.

We haven’t merged 3.10 support yet, and I wouldn’t recommend moving any production workloads to 3.10 yet until we have more confidence that the python 3.10 ecosystem that Ray interacts with is stable.

1 Like
13

Ah nice to hear @Alex thank you.

Looks like there are no Docker images for 3.9 though. Should I open up a ticket for that?

14

hmmm I think we are?

https://hub.docker.com/layers/rayproject/ray/1.11.0-py39-gpu/images/sha256-f85ce67853a85b5e094b4aed434cee938daea3fa1935fb8be690ef3253ace6c3?context=explore

15

Oh OK. We have traditionally been using rayproject/ray-ml which stops at py38.

But let me try rayproject/ray thanks.

16

ah yes, there are some ML dependencies that hadn’t supported 3.9 yet. The issue is being tracked here, and this is as good of a time as any to revisit the issue :slight_smile: