I plan to increase stability and performance and to decrease the cost of maintenance in the Photonai code base. I add clustering functionality to the original code base and change the architecture.
A small code refactoring task is usually fixing bugs. Some consider that it is not refactoring if the bug-fixing occurs before releasing to test.
A significant code refactoring project example is causing a program to be Y2K-compliant but not changing functionality. Y2K compliance is enabling code to operate correctly with dates at or beyond January 1, 2000. (Yes, this was a thing!)
Python is a dynamically typed language. However, starting with Python 3.5 (PEP 484), type hints were introduced. Type hints (note: not strong type checking) make it possible, post coding of Python, to do static type checking of code.
Here’s a great figure showing the evolution of Python type hinting:
The rendering of high-quality architecture diagrams of Azure, AWS, and GCP is shown using the Python package Diagrams. Diagrams depend on the Graphviz runtime. This article shows step-by-step how to create a Docker image with Diagrams and Graphviz. All code is included and can be downloaded.
I have posted several articles on how to create development and test Docker images [see references 4, 5, and 6 below]. I assume you know of Docker and have read them.
Docker is used for encapsulating an individual image of your application.
Docker-Compose is used to manage several images at the same time for…
A game that involves decision making is poker.
There is a lesson I had to relearn — twice.
Have you ever heard the expression?
Look before you leap. -Anonymous
Back in 1986, before the Web and the Goggle-search-bar, I “invented” a package that I ported to several different languages (FORTRAN, C, and Lisp).
I called this package “Remote-Eval.”
A few months later, the local Sun Microsystems sales rep demonstrated the incredible new “workstation,” Sun-3.
So far, so good. Until the Sun Microsystems sales rep demo-ed NFS, he said, and I paraphrase:
It is accessing the filesystem on my remote office server over the network. …
You can use the Jupyter Notebook on your local computer. Google Colab improves on the Jupyter Notebook in many ways. Here are the seven most powerful reasons to use Google Colab:
.ipynb
file to the Google Drive associated with the Colab login. It is helpful to have a separate Google account for each project and thus a different Google Drive.Note: You can create a Git account for any project folder on Google Drive. Each team member hosts on a variety of different local…
You need to use Python 3.7 or greater as @dataclass
decorator was added in Python 3.7.
You can create an isolated version to experiment using docker. You can read about my docker solution in these blog articles:
The following section explains a docked turnkey solution for you.
You can use Python 3.7, 3.8, or 3.9 in a Docker image by adding the following commands to your ˜/.bashrc_profile
or ˜/bashrc.txt.
devdir='<path-to-projects>/photon/photonai/docker/dev/'
testdir='<path-to-projects>/photon/photonai/docker/test/'echo $devdir
echo $testdirexport testdir
export devdir#
alias updev="(cd $devdir; docker-compose up) &"
alias downdev="(cd $devdir; docker-compose down) &"
alias builddev="(cd $devdir; docker-compose build) &"# alias…
In your work as a Data Engineer or Data Scientist, you will spend a great deal of your time pre-processing data to accomplish practical training of your MLm and then accurate predictions from your MLm.
Before data is input into the Machine Learning model (MLm), there may be some benefit from datatype conversion and other transmogrifications.
The process of changing data to be better consumption for the MLm is called data pre-processing.
I focus on datatype conversion for Machine Learning, as well as how to create pandas functional statements using function chaining and placement of these functions in the pyjanitor…
I’ve pulled together these techniques from 35 years of programming in multiple languages on multiple projects. There are before and after Python code examples that apply each technique.
The techniques divide into five categories:
But first, I’ll introduce the projects or code repositories we’ll use to apply these techniques.
PHOTONAI incorporates scikit-learn and other machine learning (ML) or deep learning (DL) frameworks with one unifying paradigm. PHOTONAI adopts scikit-learn’s Transformer
class method architecture.
PHOTONAI adds code that reduces manual coding and errors by transforming pre- and post-learner algorithms…
I show ten visual animations of PyCharm IDE for creating a new project or enhancing an existing project.
I am a visual learner. I learn quickly and remember longer if you show me how to do something with a visual animation rather than describe it with text. Enclosed, you find ten visual animations of PyCharm to create a new project or enhance an existing project.
We use Python predominately (90%) over the last seven years because: