Developing and organizing efficient code after preliminary experiments in machine learning [ML] requires an IDE. This mini-series of articles deals with the setup of a Python environment which supports Eclipse – and Jupyter notebooks. A key ingredient is “virtualenv”: it defines an encapsulated environment for a particular python interpreter together with a specific collection of library packages. In the last article
Eclipse, PyDev, virtualenv and graphical output of matplotlib on KDE – I
we prepared such a virtual Python3 environment “ml1” at a path “/projekte/ai/ml1” and installed some of the basic ML packages there with the help of the “pip3”-mechanism. Within Eclipse we implemented the PyDev plugin. During the setup of a “Python project” we could refer to our “ml1“-environment by defining paths to the Python interpreter and library packages located there.
Changes of the PYTHONPATH from Eclipse/PyDev
To integrate our future own Python modules into interactive experiments we need to add the paths to our own Python file directories into the PYTHONPATH variable. We expect that this should be possible from within Eclipse – and indeed it is on the project level.
In the left Eclipse view of the “PyDev explorer” we add an example directory “mytestcode”; we do this by a right-click on “ml1″ >> New >> folder” and giving the new folder a name in the eventual popup
As soon as the new folder appears we right-click on the root folder of our project “ml_1” in the PyDeev package explorer; in the appearing window we click on “Properties” and get:
There, we choose “PyDev – PYTHONPATH”. By clicking on the button “Add source folder” we can add a folder, e.g. “mytestcode”.
From now on we can import modules in any interactive Python command environment from this directory.
Python console in Eclipse
To perform experiments within an IDE as Eclipse we need some interface to interactively run Python commands and programs. A basic interface for this purpose is a “console”. PyDev, of course, offers a special Python console. How to start it?
If you have chosen a Python perspective within Eclipse you may already see a view area with a console. We start, however, from a perspective where no console view is open, yet:
To add the console view area we use the menu point “Window >> Show View >> Console”.
This gives us:
n
We got a “Debug console” – not exactly, what we want right now. So, let us open a new console view:
Again a debug console – but we change this now to a PyDev console:
At last, we get a popup where we can choose between a number of defined Python interpreters for command execution. You should at least see 2 items here: A reference to the Linux-system’s Python installation’s interpreter plus a reference to the interpreter configuration of the virtual Python environment, which we had set up in the last article. We had given it the name “python_ml1”.
We chose it; in my case this results in the following view:
Ok, we have a Python prompt (>>>) – but a bunch of error messages, too… The error messages indicate that something to access the graphical environment is missing; PyDEV’s console actually has recognized that it needs an Qt5-based interface to the desktop.
The reason for this is that I had done some customization of the “PyDev” console beforehand; when you look at the choices of “Window >> Preferences” you may find something like this:
here, the setting for “Enable GUI event-loop integration” is interesting: I had chosen the option “PyQt5(qt5)” from the combobox. To me this seemed to be a natural choice on a Qt5-based KDE Linux desktop. Remember, I had the Qt5 python modules installed on my Linux system … Well, error messages nevertheless …
Does the console work at all? Can we use “matplotlib”?
We briefly test whether the Python console works at all:
Yes! And:
We actually do get a reasonable output from “matplotlib”! However, this is NOT based on a “Qt5”-backend, but “TkAgg” (which we can see by the graphical layout of buttons). Where does this come from? And why the complaint of our console about “Qt5”?
Let us try another option from the Combobox : Tkinter(tk).
And then starting yet another console:
Hey, no error messages! This is again a strong indication that some things are missing.
Enable Qt5!
A natural guess is that we need PyQt5 within our virtual environment. Have a look at the Interpreters by choosing
“Window >> Prefrences>> PyDev >> Interpreters >> Python Interpreter”.
There we find no path to the system’s directory for “site-packages”; only the path to thw “ml1”-environments site-package directory is included in the PYTHONPATH. Now, we use “pip” from within Eclipse. This can be done by choosing our “python_ml1” in the upper area and then clicking on “Packages”:
No PyQt5 there – but a button “Install/Uninstall with pip”; we confidently use it:
We terminate all our consoles, we reset the “Interactive console settings” for the GUI event loop integration” (see above) to “PyQt5” and start again a new console for our environment’s “python_ml1” interpreter:
Good! No error messages any more; and:
Yeah, that’s what we want!
Other matplotlib-settings
You should also be aware of the fact that the backend for “matplotlib” may also be defined in a specific configuration file of your environment. In my case we find the relevant file at “/projekte/GIT/ml_1/lib64/python3.6/site-packages/matplotlib/mpl-data/matplotlibrc“.
There you find a commented entry
# backend: :Agg” ,
which you could un-comment and set to a default of “Qt5Agg”. But this is only seldomly required:
Reading the information text in matplotlibrc, we see that Qt5Agg was
automatically chosen as the first working backend of a list of possible backends: MacOSX Qt5Agg Qt4Agg Gtk3Agg TkAgg WxAgg Agg.
By the way this together with the information at https://askubuntu.com/questions/1045720/what-is-a-good-default-backend-for-matplotlibexplains explains why TkAgg worked.
Console colors and command history
Via “Window >>: Preferences >> PyDev >> Interactive Console” we can adjust the console colors. I use the following settings to get a dark background:
Command history: The PyDev console, of course, also allows for scrolling through commands but the arrow-up/down-keys. The number of commands can be set via the option “Maximum number of lines to store in global history …”.
Conclusion
A basic Eclipse/PyDev environment which supports a “virtual Python environment (virtualenv) and graphical output in Qt5 quality is set up quickly. We can use it as a tool to collect, rectify and optimize code of experimental Jupyter sessions in Python source files.
In the next article
Eclipse, PyDev, virtualenv and graphical output of matplotlib on KDE – III
we shall have a brief look at debugging local Python code in PyDev.