python34 for Amazon Linux is Missing Its Wheels
Since writing my first two posts I found out that I was doing it wrong. In the past I would use virtualenv to create virtual environments. I would also use the activate script which I later found out "drops turds" into your shell environment. All of this was a result of my upbringing in Python, and I suspect that many people still have this issue to some degree. They use what they know works and stick with it.
It seems that Amazon is no different.
Doing it Right
The proper way to create a virtual environment in Python 3.4 is to use the following
export VENV=~/env python3 -m venv $VENV
It turns out that Python has everything you need to create a virtual environment, as long as Python is built correctly. Also, avoid the activate script. Instead, use your newly created environment variable to invoke parts of your virtual environment directly. For example:
It isn't Amazon Linux at Fault
Amazon Linux is derived from Red Hat, and Red Hat has a guideline of not bundling system libraries. Unfortunately this meant that a decision was made to not include the pip and setuptools wheels within the ensurepip package.
As a result, trying to create a virtual environment with the commands above are broken, and return the following error:
Error: Command '['/home/ec2-user/env/bin/python3', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1
Long Story Short
I have since posted on AWS's forums to get them to include a python34 build that does have the wheels. Hopefully, they do, but it's possible that they won't or that it will take them some time to get AMI's that include the proper Python install. There are work arounds, but they aren't pretty.
I am going to avoid retelling the whole story here, but you can follow my adventure, the bug on Red Hat, my post on the AWS forum, or the stackoverflow thread that led me to the Red Hat bug. Let's hope that this get resolved quickly and that I can remove the workarounds from my How-Tos.
Special thanks to the following people for providing support to help solve what was really happening
- Steve Piercy
- Bert JW Regeer
- Tres Seaver
- Michael Merickel
This site/blog/soapbox is where I intend to post stuff that I feel might help someone even if the only someone is me. For now I only plan to post how-tos and maybe an occasional response to some new tool or workflow.
The site is still being tweaked, so please be patient. As time goes on I will add new features.
In real life, I am a IT manager who started as a PeopleSoft consultant out of college, but then stumbled into Python via PHP (funny story). Since I learned quite a bit about databases in my Peoplesoft days I kept it up by making sure the databases behind the apps I maintained stayed healthy. I eventually landed at BenefitPlan Manager where I am trying to make things a little better each day.