COM Port Name Pop-up

When working with microcontrollers, especially helping students debug their work, I hate having to open device manager or a serial terminal to search for the serial port name of the device I most recently plugged in.  I wrote a small python script to hook into Window’s notification system to know when a new COM port has been plugged in, and then display its name in a popup window.

The first step in this was to find out when something is plugged in.  I tried all sorts of things with windows event viewer to see if I could detect this because I didn’t want to have yet another service running in the background on my PC eating resources all the time, and I wanted the reaction to be fast, so polling every 10 seconds or something would be too long to wait for a popup.  Alas, I couldn’t find any way to get USB port data to trigger an event in windows when you plug in a USB port.

Looking online for examples I only seemed to find C# or VS code.  I’m not installing anything more to develop this one project, and at first I wanted to see if there was a cross-platform solution since I was going to have to write it all from scratch.  Eventually I settled on a windows-only solution because I found an excellent site that is full of good code examples for win32 python interfacing from a user’s post on stackoverflow. these kinds of sites are few and far between, so I’m glad I came across it!  This code example connects into Windows Notification center and WMI (Windows Management Interface) to detect when USB items are plugged in. You can detect drives, networking, and serial ports based on the example in the link.

Once this info is gathered, the next step is to create a popup with the COM port name. For this I came across an example using the plyer library for python which is dead-simple.

When you run python code, by default, the command window is always visible, so I tried a few things to hide it. Firstly, you should be able to just use pythonw.exe instead of python to run the code and it should hide the window. For some reason, this didn’t always work. The next thing I tried was using the win32 library in python. This grabs the foreground window and hides it. Since the foreground window is likely this python script, it works fine. Running it sometimes takes a faction of a second for it to hide the window so sometimes you see it flash a command window.

The final step is to have this app start when you turn on the computer. Some folks said you could just put your python file into the windows startup folder, but this never worked for me. Instead, I keep the python script in my installations folder (where I keep all my useful apps) and I wrote a batch script to run the python code. I saved the batch script into my startup folder and rebooted.  It works great and doesn’t use many resources at all!

Get my python code and batch script here.  If you use it, let me know in a comment.

Now,  if you use linux or mac, I recommend you check out usb-ser-mon which has some of the same functionality of detecting plugged-in devices. Then you can use the same plyer code to generate a popup from what I understand.

Making Procedurally Generating Spalting Patterns

I did this a while back, but someone had an interest in this on hackaday so I tried to dig up my old code.

The goal is to generate wood grain patterns for laser cutting/CNC of plywood. Mainly, I wanted to have something similar to spalted maple laser etched onto the plain birch veneered plywood. The thing that inspired this project was this post of speaker boxes on imgur I came across. The design looked really cool, and while spalted maple is expensive, birch plywood is relatively cheap.

My first step was to procedurally create a random pattern.  Similar to the imgur post, I looked at creating camouflage. I found a great example code on openprocessing by ThingOnItsOwn that used perlin noise to create a camouflage pattern. I tweaked the values a bit experimentally, then stretched the entire design to make the final design look more like woodgrain.

The next step was to just capture the edges of these blob shapes from the first pattern. I came across this example from Richard Bourne It is forked from this example from R. Luke DuBois.   Honestly I was being lazy because I had written edge detection code in college as it is standard image processing, but I knew someone else had it already in processing. Instead of using for loops, this version manually calculates out the kernel.

This leaves me with a result that looks pretty realistic.

Camo:

Spalting:

This creates a PNG filetype which can be used to add texture to a 3D print in the slicer. The slicer will adjust the print to incorporate the texture in 3D giving it a woodgrain-like effect. You technically could use this as-is on a laser cutter to create spalting like my inspiration, however being raster data, it would take the laser a long time. It’d have to scan the laser (the thickness of a human hair) across the entire area of the panel you are applying the texture on. To make this faster, you can vectorize the PNG in inkscape or other software to outline the dark areas of the PNG.  This will cut fast as it is a vector (the laser would just have to draw the lines the same way your hand would. That would save a lot of time.

If anyone wants to add vectorizing to my code, please do! You can clone my github and put it on P5.js. I was going to add it, but I got lazy again. I even asked ChatGPT3 to help combine this code with something like potrace or imagetracerjs but it produced code that looked great, some even compiled after a few tweaks, but never worked.

Ultimate List of Tips, Tricks, and Tutorials for Fab Lab Students

This post is a not on a lot of techniques for using different types of machines and processes for making stuff. I’ve had this for years but decided to finally publish it. I’ll add to it periodically, but I figured others might find it useful as well. RIGHT-CLICK to open in new windows.

Laser Cutting:

Embedded Systems, Microcontrollers, and Arduino

Circuit Board design and Fabrication:

3D Printing:

Casting and Mould Making:

CNC:

Machines:

Mechanical:

Metal-Bending:

Miscellaneous:

 

 

================================================

My favorite Fabrication-related researchers:

Dr. Stephanie Meuller at MIT’s research group

Dr. Patrick Baudisch at Hasso Plattner Institute Human computer interaction reserach group

 

Teaching Research:

Sketchnoting basics

Graphic Recording

Sketchnote travel journal to get started

Control Theory:

Brian Douglas’s awesome youtube channel explains Control with some great examples.

Kat Kim has another great channel on Controls as well as other Electrical and Computer engineering examples and lectures

George Gillard has a great whitepaper explaining PID controls

Another great PID example is from this Reddit thread

Learning Math concepts:

MathVault – Learn higher-level (college-level) math concepts more intuitively

BetterExplained.com ADEPT model for learning math intuitively

Good sources of materials:

XXXXXXXXXXX    Todo when I’m not so busy or lazy: XXXXXXXXXXXXX

Add sections for PCL shapelock and other named plastics to ultimate FabLab list.

Also add cardboard modeling guy and nibbler tool

Add anodizing alum and titanium, bluing/blacking steel,

And interesting research I like with lasers  hydrographics and uv printers and metal hologram art

How to fix MPLabx ‘s Errors

I recently taught a class utilizing PIC chips and MPlabx. This is the first time I used PIC chips in about 20 years. I have to say, it has been one of the most frustrating experiences I have had programming chips in a very long time.  MPlabx is a modified version of Netbeans, with which I am intimately familiar with as I used it for my graduate work. My class had an impossibly hard time consistently programming chips. This page documents some of the ways we fixed the problem.

While coding, sometimes, seemingly randomly, MPLabx will start throwing underlining things and start throwing errors such as “TRISA no defined.”  It is as if the link to the libraries gets severed. I have no clue why this happens, but two things seem to fix it.  1. Save the file you are editing because sometimes it is just that the changes you’ve made have not been saved and it can’t find some of the declarations for things. 2. When the problem persists or actually won’t compile, then you have to Restart MPlabx.  Once it loads up again, it will re-register all the library paths and all is well. If this doesn’t fix the problem, then check the errors and make sure you have the correct number of  curly bracket “}” and in the right places.  Remember, no executable code can exist outside of functions except global declarations.

 

Programming issues:

One of the biggest issues was powering the circuit. By default, every project expects your circuit to be powered externally.  The programmer even has a pin dedicated to sensing this voltage and stopping everything if it isn’t sensed.  You have 2 options. Either power the circuit externally, or you can go to project properties, Conf–>PIckit4 (or whatever programmer you have)  and select “power” from the dropdown and checking to power the board. Check the box next to “Power target from…”

This will only work if your circuit is low-powered. If you need a significant amount of power, then you must power the board separately and make sure this setting is NOT checked. Sometimes this can prevent the chip from programming, but more on this later.

Programming Pins:

The programmers only have a few pins that are used on the processor, but those are very important.

Memclr/Reset: This pin basically tells the processor it is being reprogrammed. (There’s more to it, but out of the scope of this post).

Target voltage: This senses whether or not the processor has power. If not, then the programmer won’t attempt to talk to it and will instead error out.

PGD: Programmer data.  This is often on a standard port pin, so if programming keeps failing, isolate this pin form anything else. ie. remove all externally connected things like buttons, LEDs, whatever else you might have connected to this pin and only connect the programmer.

PGC:  Programmer clock. The same applies from above.

Gnd: Communication required a shared ground so make sure this is connected correctly.

If the PGD, PGC, and GND are not connected correctly, you’ll often get the following type of error:

Target Device ID (0x0) is an Invalid Device ID. Please check your connections to the Target Device.

Each processor has a special register that contains a number identifying what type of chip it is. The programmer requests this info before programming so it won’t accidently brick the chip by writing to it in the wrong places. The device ID is NEVER 0x0 for any processor. If you see this error that means that the programmer was unable to get a correct value from the PGD pin. Check your connections for shorts, opens, and remove any circuitry attached to the programming pins except the programmer itself.

Otherwise, you might get something like the following:

Target Device ID (0x6750000) is an Invalid Device ID. Please check your connections to the Target Device.

This error shows that a value number was returned, however this doesn’t match the chip your project is set to compile for.  Check your program properties and make sure the chip number matches the part number labeled on the processor itself.

Programmer Quality:

While not do or die, I will say that a good programmer is helpful.  Don’t get me wrong, I’ve cobbled together some super janky programmers back in the day that have worked, but having a quality device certainly helps troubleshoot.  We originally started using PICkit3 programmer (without the Microchip logo, which I think indicates they are clones).  For about 2 weeks, while it worked about 80% of the time on my laptop, having a class of laptops with different configurations ,etc. led us to have such problems that I searched for other solutions.  The first thing that helped was to use the MicrostickII and solder up a converter board that converted the programming pinout to a programing cable.  This solved the problem for a few students.

Eventually I was approved to buy official PICkit4 programmers, which seemed to help even more.  However unexpectedly and randomly, some folks still had a warning pop up suddenly, after successfully programming multiple times in a row, which prevented them from having success.  There would be absolutely no hardware or settings changes, only small tweaks to the code, but suddenly the chips would refuse programming.

Though there were multiple error, the most common was

Programming did not complete.
You are trying to change protected boot memory. In order to do this you must select the “Boot, Secure and General Segments” option on the debug tool Secure Segment properties page.

Attempt 1 (rarely successful): Some folks online suggested manually erasing the chip first, then trying to program.  This rarely works.

Attempt 2 (sometimes successful): Maybe your circuit requires more power than your programmer or USB can provide. Go in the settings as described above and uncheck “Power target circuit…”  and make sure to provide an external power source.

Attempt 3 (sometimes successful): Again, this is after multiple successful attempts and no hardware or program setting changes.   My predecessor told me that when this happens, the fix is typically to do the following:  Open the project properties, Select Conf–>PicKit4 (or whatever programmer you have) then select “Memories” from the dropdown and change it from “Automatic” to “Manual”  then click “Apply”  Then click it back to “Automatic” and apply then click OK.  This seems to work sometimes

Attempt 3 (the actual fix): Well, I could have just read the error message because it tells you just what to do… kind of.  Open project properties: –>Conf->PICkit4 then from the dropdown, select “Secure Segment” and change the “Segments to be programmed” to “Boot, Secure and General Segments”

Why is this not default?   Why does it work multiple times, then suddenly require this change?  Who knows?  I would absolutely love an explanation for this behavior.

 

 

After successfully programming for a n hour or so, a new error I inexplicably received suddenly appears as follows:

Erasing…
[config mem] 0xf80000, expected 0xcf, got 0x0.
You have set the program speed to Normal. The circuit on your board may require you to slow the speed down. Please change the setting in the tool properties to low and try the operation again.
Programming did not complete.

Changing programing speeds does nothing:

Erasing…
[config mem] 0xf80000, expected 0xcf, got 0x0.
Programming did not complete.

If you run into this, disconnect power to your chip and programmer, and plug it all back in. Sometimes this requires you just unplug your programmer from USB and replug the programmer to your computer.   Again, this happens when I’ve been successfully programming for an hour or so, literally nothing changes except tweaks to my code, and suddenly nothing wants to work. No explanation for as to why this happens.

Adding Custom Desktop and Taskbar Icons to you Python Executable

After going through the process of setting up pyinstaller to stop throwing false virus warnings, you probably want your fancy new desktop app to have its own custom icon.  There is a special method required for each of these icons to work as you expect.

Normal windows programs are a single .EXE file with a custom icon. When you open them, there’s a GUI and the taskbar and window icons are the same. To accomplish this, I use pySimpleGUI to design the GUI. This is simple to install and there are several version available such as pysimpleguiQt. Install this in the command line with:

pip install pysimplegui

I actually cheat a little bit here. You can do everything on the command line, but pysimplegui has a great GUI for this that simplifies things called pysimplegui-exemaker.

Step 1: Create a folder structure for your python project.

There’s likely easier ways to do this, but I typically start my projects by visiting github.com and creating a new repository on my account. Then I open github desktop and clone that directory to my computer. This gives me a correct structure for keeping track of any changes in that folder. I’ll create my main python file in this folder and edit the README file with a description and my project goals.

I create a folder to hold my assets named something like “assets” or “img” where I will save my custom icon file. This can be edited later, but I like having something here so I know from the beginning that it is set up correctly.

Step 2. Get or create a .ico file.

This is the image of your icon. They are really easy to find free ones online or you can make one from scratch using a free online editor or some image editors gimp.

Step 3. Setup Relative Filepaths

I add the following to the top of my main python script based on advice from arcade academy which allows relative filepaths for pyinstaller:

if getattr(sys, 'frozen', False) and hasattr(sys, '_MEIPASS'):
os.chdir(sys._MEIPASS)

Step 4: Setting the Window Icons:

To get my custom icon in the pysimpleGUI you have to use a fill absolute path when you create your windows and all popup windows. I typically just create a global variable with the absolute path to the location of the icon.

program_icon = C:\Users\ALaptop\Documents\GitHub\myProjectFolder\assets\icon.ico
window = sg.Window('Program Title Goes Here', layout, icon =program_icon)
filename = sg.popup_get_file('Select a MPA file', no_window=True, file_types=(("myPythonApp Files", "*.mpa"), ("Text files", "*.txt"), ("All Files", "*")), icon =program_icon) 

When you run this code as a script (in VScode for example) you should see the custom icon at the top left of all the program’s windows. But your taskbar will still be a generic python icon.  You won’t see the icon in the windows taskbar until you create and run an executable using pysimplegui’s exemaker or pyinstaller. Simply running the script from the command line will NOT show the correct icon in the taskbar.

Step 5: Setting Desktop and Taskbar icons

Pyinstaller can set up a template for us to fill out for our project so it will be able to bring in all the external files we want (like icons). Simply run the following command in the command line to generate the template:

pyinstaller --onefile --noconsole .\my-Program.py
This creates a new file called “my-Program.spec” in the same folder as your code. Open this in a text editor and it’ll look something like this:
# -*- mode: python -*-

block_cipher = None

a = Analysis(['my-Program.py'],
pathex=['C:\\Users\\ALaptop\\Documents\\GitHub\\myProjectFolder'],
binaries=[],
datas=[],
hiddenimports=[],
hookspath=[],
runtime_hooks=[],
excludes=[],
win_no_prefer_redirects=False,
win_private_assemblies=False,
cipher=block_cipher,
noarchive=False)
pyz = PYZ(a.pure, a.zipped_data,
cipher=block_cipher)
exe = EXE(pyz,
a.scripts,
a.binaries,
a.zipfiles,
a.datas,
[],
name='my-Program',
debug=False,
bootloader_ignore_signals=False,
strip=False,
upx=True,
runtime_tmpdir=None,
console=False )

Step 6: Customizing the .spec file

Once the .spec file is created, you’ll want to edit it. In the Analysis section in datas I put the links to any assets the program will need such as images, sounds, etc. If I only have images in a subfolder named “img” it would be

datas=[( 'img/*png', 'img' )],

The first parenthesis is where the files are currently stored, and the second is the name of the folder inside the temporary folder that pyinstaller uses when the exe runs. Remember how I said Pyinstaller creates a self-extracting zip file? Here’s where that comes into play. You are telling pyinstaller where you stored your assets relative to your project’s folder, and then telling it where it can put these files later. This path is important especially if you used relative links in your python code. If your EXE can’t find the files it needs, it’ll crash giving you a popup stating “Failed to execute script my-Program”. This can be confusing to debug if you don’t know how to see what the executable is looking for. Also, this expects that you pasted the lines from Step 3 above into your code which sets up relative filepaths.

To understand and to debug this, you have to remember that the EXE pyinstaller creates is a zip file that is made up of a bootloader (which can actually run your python code), your python script, any libraries your script uses auto extracts a folder, and any other files needed to run your script (images, audio, etc). When you run the executable, it actually decompresses all this and sticks it in a folder in your C:\Users\<username>\AppData\Local\Temp in a directory named “_MEIxxxxx” where the ‘x’s are a random number. This folder exists only while the executable (or the error popup) is opened. As soon as you close it, this folder is deleted. You need to visit this folder to debug issues. To debug, you’ll need to pay close attention to the paths in the spec file and your code’s paths and then comparing them to the location of assets inside this temporary _MEIxxxxx folder.

For instance, in the datas entry above, the second set of quotes denotes the name of the folder INSIDE the temporary folder. That’s where pyinstaller is placing the PNGs I refer to in that line. If my app keeps crashing for some reason, I can make sure that these files exist in the correct folder in the C:\Users\<username>\AppData\Local\Temp\_MEIxxxxx\img folder.

Add the absolute path to your custom icon to the .spec file. This is added to the “EXE” section as part of the list of settings. I also added “console=False” so it would not open a console while executing.

console=False , icon='C:\\Users\\ALaptop\\Documents\\GitHub\\myProjectFolder\img\\icon.ico')

You can see my full spec file below. Once you’ve saved the spec file changes, you will issue the same command as before, but use the .spec file instead of the python file.

pyinstaller --onefile .\my-Program.spec

Just for good measure, I also put the fullpath in the pysintaller command as well sometimes:

pyinstaller --onefile --noconsole --icon "C:\\Users\\ALaptop\\Documents\\GitHub\\myProjectFolder\\icon.ico" --add-data "*png;." .\my-Program.spec

Here’s the full final .spec file I would use:

 # -*- mode: python -*-

block_cipher = None


a = Analysis(['my-Program.py'],
             pathex=['C:\\Users\\ALaptop\\Documents\\GitHub\\myProjectFolder'],
             binaries=[],
             datas=[( 'img/*png', 'img' )],
             hiddenimports=[],
             hookspath=[],
             runtime_hooks=[],
             excludes=[],
             win_no_prefer_redirects=False,
             win_private_assemblies=False,
             cipher=block_cipher,
             noarchive=False)

             
pyz = PYZ(a.pure, a.zipped_data,
             cipher=block_cipher)
exe = EXE(pyz,
          a.scripts,
          a.binaries,
          a.zipfiles,
          a.datas,
          [],
          name='my-Program',
          debug=False,
          bootloader_ignore_signals=False,
          strip=False,
          upx=True,
          runtime_tmpdir=None,
          console=False , icon='C:\\Users\\ALaptop\\Documents\\GitHub\\myProjectFolder\\img\\icon.ico')

 

Program Icon (Desktop Icon):

This will be the icon users see and doubleclick on their desktops to open your app.

After running pysimplegui’s exemaker one time for your project as normal, it will create a .spec file template for you.  Once the .spec file is created, you have to put the absolute path to the icon you want to use in that .spec file. For instance, I had my icon in a folder called “img” inside my python’s project folder. Paste the path at the bottom of the EXE section like this:

console=False , icon='C:\\Users\\ALaptop\\Documents\\GitHub\\myPythonApp\\img\\icon.ico')

Just for good measure, I also put the fullpath in the pysintaller command as well:

pyinstaller --onefile --noconsole --icon "C:\\Users\\ALaptop\\Documents\\GitHub\\myPythonApp\\icon.ico" --add-data "*png;." .\myPythonApp.spec

Window and Taskbar Icon:

This is set in pysimgplegui when you create any window (even popups can be given a different icons). The only way this works is if you code the full absolute path in the python script, but you won’t see it in the taskbar until you create and run an executable using pysimplegui’s exemaker or pyinstaller. Simply running the script from the command line will NOT show the correct icon in the taskbar.

program_icon = 'C:/Users/ALaptop/Documents/GitHub/myPythonApp/img/icon.ico'

filename = sg.popup_get_file('Select a MPA file', no_window=True, file_types=(("myPythonApp Files", "*.mpa"), ("Text files", "*.txt"), ("All Files", "*")), icon =program_icon)