Sign in

Physicist, Machine Learning Scientist and constantly improving Software Engineer. I extrapolate the future from emerging technologies.

Let’s expand the definition of refactoring to include new functionalities

coffee cup beside a keyboard and computer screen
coffee cup beside a keyboard and computer screen

Different Kind of Refactoring

I plan to increase stability and performance and to decrease the cost of maintenance in the Photonai code base. I add clustering functionality to the original code base and change the architecture.

A small code refactoring task is usually fixing bugs. Some consider that it is not refactoring if the bug-fixing occurs before releasing to test.

A significant code refactoring project example is causing a program to be Y2K-compliant but not changing functionality. Y2K compliance is enabling code to operate correctly with dates at or beyond January 1, 2000. (Yes, this was a thing!)

“Refactoring is a disciplined technique for…


Let’s discuss why type hinting techniques and tools improve your Python code

old-fashioned type in a case
old-fashioned type in a case

Overview of Type Hints or Type Annotations

Python is a dynamically typed language. However, starting with Python 3.5 (PEP 484), type hints were introduced. Type hints (note: not strong type checking) make it possible, post coding of Python, to do static type checking of code.

Here’s a great figure showing the evolution of Python type hinting:


Visualize your architecture

clouds seen from above
clouds seen from above

The rendering of high-quality architecture diagrams of Azure, AWS, and GCP is shown using the Python package Diagrams. Diagrams depend on the Graphviz runtime. This article shows step-by-step how to create a Docker image with Diagrams and Graphviz. All code is included and can be downloaded.

Docker Solution for Graphviz, Diagram, and Cluster

I have posted several articles on how to create development and test Docker images [see references 4, 5, and 6 below]. I assume you know of Docker and have read them.

Docker is used for encapsulating an individual image of your application.

Docker-Compose is used to manage several images at the same time for…


There is a lesson I had to relearn — twice.

Have you ever heard the expression?

Look before you leap. -Anonymous

Back in 1986, before the Web and the Goggle-search-bar, I “invented” a package that I ported to several different languages (FORTRAN, C, and Lisp).

I called this package “Remote-Eval.

A few months later, the local Sun Microsystems sales rep demonstrated the incredible new “workstation,” Sun-3.

So far, so good. Until the Sun Microsystems sales rep demo-ed NFS, he said, and I paraphrase:

It is accessing the filesystem on my remote office server over the network. …


Google Colab can access any public Jupyter Notebook from GitHub or Drive

Why Use Google Colab?

You can use the Jupyter Notebook on your local computer. Google Colab improves on the Jupyter Notebook in many ways. Here are the seven most powerful reasons to use Google Colab:

  1. You can get any public Jupyter Notebook from a GitHub repository.
  2. You load, edit, and save any .ipynbfile to the Google Drive associated with the Colab login. It is helpful to have a separate Google account for each project and thus a different Google Drive.

Note: You can create a Git account for any project folder on Google Drive. Each team member hosts on a variety of different local…


We show how @dataclass transforms Python code into better Python code with twelve before and after code examples.

Upgrade to Python 3.7 or later

You need to use Python 3.7 or greater as @dataclass decorator was added in Python 3.7.

You can create an isolated version to experiment using docker. You can read about my docker solution in these blog articles:

The following section explains a docked turnkey solution for you.

Dockerize

You can use Python 3.7, 3.8, or 3.9 in a Docker image by adding the following commands to your ˜/.bashrc_profile or ˜/bashrc.txt.

devdir='<path-to-projects>/photon/photonai/docker/dev/'
testdir='<path-to-projects>/photon/photonai/docker/test/'
echo $devdir
echo $testdir
export testdir
export devdir
#
alias updev="(cd $devdir; docker-compose up) &"
alias downdev="(cd $devdir; docker-compose down) &"
alias builddev="(cd $devdir; docker-compose build) &"
# alias…


Includes all Python code, offers pandas chaining, and extends the pyjanitor package.

Introduction to data pre-processing: Datatype Conversion

In your work as a Data Engineer or Data Scientist, you will spend a great deal of your time pre-processing data to accomplish practical training of your MLm and then accurate predictions from your MLm.

Before data is input into the Machine Learning model (MLm), there may be some benefit from datatype conversion and other transmogrifications.

The process of changing data to be better consumption for the MLm is called data pre-processing.

I focus on datatype conversion for Machine Learning, as well as how to create pandas functional statements using function chaining and placement of these functions in the pyjanitor…


Tips for Python documentation, coding, testing, verification, and continuous integration

Glass orb with patterns
Glass orb with patterns

I’ve pulled together these techniques from 35 years of programming in multiple languages on multiple projects. There are before and after Python code examples that apply each technique.

The techniques divide into five categories:

  • Documentation techniques
  • Coding techniques
  • Testing techniques
  • Verification techniques
  • Continuous integration (CI) techniques

But first, I’ll introduce the projects or code repositories we’ll use to apply these techniques.

What’s PHOTONAI?

PHOTONAI incorporates scikit-learn and other machine learning (ML) or deep learning (DL) frameworks with one unifying paradigm. PHOTONAI adopts scikit-learn’s Transformer class method architecture.

PHOTONAI adds code that reduces manual coding and errors by transforming pre- and post-learner algorithms…


I show ten visual animations of PyCharm IDE for creating a new project or enhancing an existing project.

I am a visual learner. I learn quickly and remember longer if you show me how to do something with a visual animation rather than describe it with text. Enclosed, you find ten visual animations of PyCharm to create a new project or enhance an existing project.

Python

We use Python predominately (90%) over the last seven years because:

  • Almost all new machine learning models, cloud, GPUs, and many other platforms are available as a Python package.
  • The assortment and number of free…

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store