Using MPlabX to Program Arduino Boards in Straight C/C++

I wanted to use MplabX to replace the graphical front-end of the Arduino app and write my own bare metal C/C++ code, but use the arduino’s built-in USB programming function. The method below can be done in multiple different IDEs, but I show how to set up MplabX. It is important to note that this method expects the Arduino’s bootloader to be stay on the board.

1. Setting up the Path to the Arduino IDE’s Compiler:

These are different for every computer so you have to find it yourself. To do this, open the Arduino IDE , go to File–>Preferences and click “Show Verbose Output” for both Compile and Download.

Next, plug in a board and send an example project to your Arduino (such as Blink.ino). The command line will spill over with commands.

Scroll to the top of the command window in the ArduinoIDE and look for the line after it says “Compiling Sketch”:

Copy this entire line and paste it to a text file so we can edit it. (Double-click this line to highlight it to copy). Here’s mine:

"C:\\Users\\adamp\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\avr-gcc\\7.3.0-atmel3.6.1-arduino7/bin/avr-g++" -c -g -Os -w -std=gnu++11 -fpermissive -fno-exceptions -ffunction-sections -fdata-sections -fno-threadsafe-statics -Wno-error=narrowing -MMD -flto -mmcu=atmega328p -DF_CPU=16000000L -DARDUINO=10819 -DARDUINO_AVR_NANO -DARDUINO_ARCH_AVR "-IC:\\Users\\adamp\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\avr\\1.8.6\\cores\\arduino" "-IC:\\Users\\adamp\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\avr\\1.8.6\\variants\\eightanaloginputs" "C:\\Users\\adamp\\AppData\\Local\\Temp\\arduino_build_752190\\sketch\\Blink.ino.cpp" -o "C:\\Users\\adamp\\AppData\\Local\\Temp\\arduino_build_752190\\sketch\\Blink.ino.cpp.o"

 

The highlighted portion above is the file path to the C++ compiler Arduino uses. If we want to write our own bare-bones C or C++ code for the arduino, this is all we need.  Open MPlabX and select “Tools–>Preferences.  In the popup, select the “Embedded” tab.  On the left side, click the Add… button to add a new toolchain. A new popup will appear where you can paste the “Base Directory”. Paste in the highlighted part of your path only up to but NOT including the “avr-g++” part.  This will allow MPlabx to use the AVR-G++ compiler for Atmel chips.

2. Creating a MplabX project:

In MpLabx, start a new standalone project. I’m using an Arduino nano so I will choose the chip “Atmega328P” I recommend you select “Simulator” for the “Tool” entry here as it can be really handy for debugging code.

Then select the new entry “AVR-GCC” for the toolchain.

3. Set up the Programming Tool in MPlabX:

Now to take advantage of the built-in USB programming of an arduino board we need to set up the correct commands. In the Arduino IDE scroll down in the command window and find the last line of white text before the red text begins. The red text is the output from the programmer app (called avrdude.exe).  The line just before this is the command that calls avrdude with the appropriate settings.

 

C:\Users\adamp\AppData\Local\Arduino15\packages\arduino\tools\avrdude\6.3.0-arduino17/bin/avrdude -CC:\Users\adamp\AppData\Local\Arduino15\packages\arduino\tools\avrdude\6.3.0-arduino17/etc/avrdude.conf -v -patmega328p -carduino -PCOM7 -b115200 -D -Uflash:w:C:\Users\adamp\AppData\Local\Temp\arduino_build_752190/Blink.ino.hex:i

Scroll to the rightmost of the command below to see the highlighted text in the line above. This is the path to the actual hex file the programmer is sending. This changes based on where you name the MPlabx project. You need to change this file path to the following to work with any MPlabX project:

-Uflash:w:${ImagePath}

Once you make this change, simply copy this entire command and paste it into the MplabX project properties.

In Mplabx, right-click the project you want to work on, select Properties, then select “Building” from the left sidebar in the popup window. Make sure to click “Execute this line after build” and paste the entire command from above in the blank.

***BIG NOTE: If you change the port number of the board you are programming (plugging in a different board, or using this same project on a different computer) you MUST change the COM port in this option in the project or it will fail to program and give a cryptic error.

4. Testing with a Blink project:

If you have set everything up correctly, then you can right-click your project and select New–>main.c

Replace the code in this file with the following:

#define F_CPU 16000000UL //You must define the clock frequency before you can use the delay function. This should match the crystal on your board.

#include <stdio.h>
#include <stdlib.h>
#include <util/delay.h> //must use this to use the _delay_ms() function
#include <avr/io.h> //Must use this to access port and pin numbers on our chip

/*
*
*/
int main(int argc, char** argv) {

       //this is where your setup() function goes
       DDRB |= (1 << PORTB5); //Make Arduino Pin13 (AVR PortD Pin5)) and output pin

      while (1) {//this is your "loop" function
           PORTB ^= (1 << PORTB5); //Toggle the value of this pin using an XOR function with itself
          _delay_ms(100); //Change this number to change the LED flashing frequency to know if it is working or not.
        }//end while 1

       return (EXIT_SUCCESS);
}//end main

Then click the build icon and it should compile, then you should see the familiar text from the Arduino IDE. The Avrdude programming information is the red text when you turn on Verbose mode.

Troubleshooting:

If you don’t see this text, a few things might be the cause:

  1. You forgot to click the “Execute after build” checkbox in the build properties
  2. You have the wrong COM port or U:flash setting in the line you want it to execute after building.
  3. Make sure you set your project up with the correct chip and correct toolchain (AVR-GCC)
  4. You have a compilation errors. This can be hard to see, but you can make it easier

If your Arduino isn’t doing what you want it to, then you likely need to debug your code. There’s a LOT to be said with this. The easiest method is to set up a UART library so you can print text out the serial port to your computer. Search github and AVR freaks for examples you can import.

Another method is to use the Simulator in MPlabx to see what bits are being set/changed in the memory location of the Atmega chip.

A third option allows you to run the code line by line on the actual chip. This would require a proper programmer (not just the arduino’s built-in USB programming).  Here’s an example of how to set that up.

Now you can learn to write your own libraries to better understand the magic of the Arduino project. Learn some of the tricks that are used to simplify the interface for programming on this site.

 

 

Baofeng TP-8Plus, UV-13 Pro, GM-30, and similar radios

So I was at the bin store the other day and I came across a couple of Baofeng TP-8Plus radios. I’m no expert on HAM radios in general (please correct me on anything I’m wrong about on this page BTW), but I got these two for $7 each so I couldn’t beat it. It turns out, the models I picked up are now a ubiquitous hardware model with different firmware which can be used for multiple purposes.

Firstly, I can’t tell the differences, but the TP-8Plus is apparently also called the UV-13 Pro.. as well as many other models (listed below). There are different levels when talking about radios like this, HAM and GMRS. HAM radio is amateur radio and you have to pass certifications tests for the license. These radios are classified as HAM radios since their broadcast power is too high for GMRS (which is a higher powered two-way walkie-talkie band). To use GMRS band, you need a radio rated for this band and a GMRS license (no certification test required) and your whole family can use the radios apparently. The radios are too powerful as they are for this, however there’s a firmware you can get for them that limits the broadcasting power turning them into GMRS-legal radios (from what I have read..)

Where to get Firmware:

Before doing anything, I took a pic of the current firmware version as shown below. You can see this by holding the flashlight button as you turn on the radio

I was unable to figure out how to copy this ROM onto my PC, but I did find an older version of the UV-13 firmware I could use to replace it if I needed to… which I did. I wish I could find the firmware online.

Where do you get firmware? It depends on what you want your radio to do. I have it on the authority of some random dude on reddit and other that this radio’s hardware “is the same for Baofeng UV-13, TP-8, Explorer QRZ-1, TYT TH-UV88, Pofung P15UV,  Rugged GMR2, TIDRADIO TD-H5, Retevis RT85, and Radioddity MU-5, GM-30.”  Firmware for those radios should work here. They each have slightly different abilities, so it is really up to you. what you’d like it to do. Do your research to see what legal licensing steps you can take based on how you want to use your radios. Comment below if you know where to find other firmware for this radio.

For using this as a HAM radio (where you can enter in frequencies manually or scan them in the 2m band) the UV-13 firmware was easiest to find. The only version I came across online from multiple sources was UV13-Firmware-V06.01.013-20211104_01013.bin I was able to successfully install this without issue and I can still write program settings using the P15-UV software.

For GMRS walkie-talkie usage, I came across the Radioddity page with their firmware. It is important to note that after installing this firmware, I had to use Radioddity’s own version of P15-UV branded “GM-30”. The link to their firmware and programming app is here. From what I read, this firmware limits the transmit frequency making it legal to use the radio. NOTE: I am not a lawyer and I have no way to test the radio’s power with this firmware to compare, it is just what I read on the internet.

Programming:

There are 2 programming modes. One for just reading/writing programmed channels and settings, and another mode for updating firmware. There’s a trick to program firmware I describe below.

The software for the Baofeng firmware is P15-UV CPS software. I was able to find a version of it from a company called abbree.  This only works for the Baofeng firmware. I had to get my copy from the wayback machine as the link on their main download page was broken. This is different software than my older UV-5R used. If you have the GM-30 firmware installed, you must use this Radioddity branded version of the programming software.

    1. I already had a USB cable for these radios since I had an older model. Once installed you can plug it up to the radio, turn on the radio and connect. In the PUV-15 software, select “Program–>Communication Port” and select the com port. If you don’t know which this is, open windows device manager and look for the COM port that disappears when you unplug the cable.
    2. Next you want to back up and save all the channels and settings from the radio by selecting “Program–>Read from Radio” and click “OK” to read all the default settings. Then you have to click “File–>Save As.” This will save your channels and any program settings to a file on your computer.
    3. To flash the firmware, download your preferred firmware version (more on this below) and select “Program–>Tool” For this to work you must manually put the radio in programming mode by turning it off, holding down the PTT and Flashlight button as you turn it back on again.  You know you did this right when the screen does NOT come on, but the red light at the top glows dimly.
    4. Click “Upgrade” button on the app to change the firmware. DO NOT INTERRUPT THIS PROCESS or you might brick the device.
    5. Rewrite the channels and program settings by choosing “Program–>Write to Radio”
    6. If you were a genius like me and forgot to backup your settings, your menus and voice might now be Chinese.  No problem, simply change the language by scrolling through the menu that says this:

      Then pressing the menu button to edit and the down button to change it to English. Press menu again to confirm the change.

    Licensing:

    Again, I am no expert on the matter, but here’s what I learned from all the forum posts and videos I found on this radio. There is no license required to use these radios as listening-only devices. You can use them as weather radios or even emergency service scanners. If you wish to transmit at all you’ll need either a GMRS license or a HAM radio license depending on what firmware you have and how you plan to use the radio. See the comparison chart below:

    GMRS HAM
    One license covers your family. Licensing process is confusing, so watch a recent tutorial video on the process to find out the steps required. One license covers you. You can allow your family to broadcast, but you must be there with them at the time.
    No exam required Must pass a certification exam (there are different levels)
    To be used as walkie-talkies (eg. in your family) because they are shorter-range transmissions. Connect with community of HAM radio operators all over the country, world, and even on the international space station!
    Use Radioddity GM-30 firmware and their custom programming app. Use Beofeng UV-13 firmware and the P15-UV programming app.
    License last 10 years License lasts 10 years
    Cost $35 Cost $35

Ultimate List of Tips, Tricks, and Tutorials for Fab Lab Students

This post is a not on a lot of techniques for using different types of machines and processes for making stuff. I’ve had this for years but decided to finally publish it. I’ll add to it periodically, but I figured others might find it useful as well. RIGHT-CLICK to open in new windows.

Laser Cutting:

Embedded Systems, Microcontrollers, and Arduino

Circuit Board design and Fabrication:

3D Printing:

Casting and Mould Making:

CNC:

Machines:

Mechanical:

Metal-Bending:

Miscellaneous:

 

 

================================================

My favorite Fabrication-related researchers:

Dr. Stephanie Meuller at MIT’s research group

Dr. Patrick Baudisch at Hasso Plattner Institute Human computer interaction reserach group

 

Teaching Research:

Sketchnoting basics

Graphic Recording

Sketchnote travel journal to get started

Control Theory:

Brian Douglas’s awesome youtube channel explains Control with some great examples.

Kat Kim has another great channel on Controls as well as other Electrical and Computer engineering examples and lectures

George Gillard has a great whitepaper explaining PID controls

Another great PID example is from this Reddit thread

Learning Math concepts:

MathVault – Learn higher-level (college-level) math concepts more intuitively

BetterExplained.com ADEPT model for learning math intuitively

Good sources of materials:

XXXXXXXXXXX    Todo when I’m not so busy or lazy: XXXXXXXXXXXXX

Add sections for PCL shapelock and other named plastics to ultimate FabLab list.

Also add cardboard modeling guy and nibbler tool

Add anodizing alum and titanium, bluing/blacking steel,

And interesting research I like with lasers  hydrographics and uv printers and metal hologram art

Best way to make GUIs and Executables in Python for Windows, Mac and Linux

Typically when making a program, I want users to have a familiar experience: a single .EXE file instead of some weird scripts that requires other programs to run. There are a few different ways of accomplishing this using python,. On this page I’ll explain the simplest way I’ve found to get this result, and how to debug if you run into problems with this method.

I wrote about using pySimeplGui for making dead simple GUI interfaces for python code before, and even have a simple example project,. Check those two articles out for that aspect.

As for creating the .EXE file, there are several packages that will allow you to create executables of your python scripts. The reason to do this is that you don’t want end users to have to install python  (simplified usage of your app) or you don’t want anyone seeing the actual python code. The way they work is that they create a self-extracting zip file that has your code, all associated libraries (including DLLs), and most crucially, they have a small bootloader that is a python interpreter executable. There are several available such as cx_Freeze, bbFreeze, Py2App, Py2Exe, Pyinsaller, or you can create your own from scratch  (See a chart comparing them here).

There are some problems with these tools. Several of these are not currently updated. Some don’t allow you to create a “One File” solution, meaning they require additional folders of files, etc. which in my view isn’t a true executable experience from the user perspective.  For some reason, there’s a known issue with some of these which makes the resulting executable get incorrectly flagged as a virus by virus software. There are some ways around this however.

I’ve tried out a few of these and given my recent experience with PySimuleGui, and the features I wanted, I settled on pyinstaller.

PysimpleGUI has its own interface for pysintaller to simplify things called pysinstaller-exemaker.

For some reason, my resulting executables kept getting flagged with false virus warnings on windows. I tested in Virus Total which runs multiple virus scanners at once on your file. Sure enough, lots came back with false positives.

I found two ways to fix this problem. As with most things, there’s the easy way, and then there’s the right way.

The easy way requires specific versions of python and pyinstaller on your machine.

The other way to prevent false positive virus scans is to recompile pyinstaller’s bootloader from scratch. While I did this (and explain it below), it is non-trivial.

The Easy Way: Install Correct Versions:

The easiest way to not have the virus issue is to use the correct versions of pyhton and pyinstaller as described on a no-dead link on yuriss.com.  This Completely solved the problem and is SOOO simple!  Basically you just install the correct version of python (Version 3.7.4) from python.org. If getting the windows version, make sure it is the 64-bit version, other users seem to have had issues with the 32-bit version.

Once this is installed, you can use pip on the command line to install Pyinstaller 3.4:

 pip install pyinstaller==3.4

The Right Way: Recompiling pyinstaller’s bootloader from scratch

This may be needed for future version of python or pyinstaller. Since I already went through the headache of figuring out how to do it, I’m documenting it here.

If using a version of pyinstaller that throws the virus warnings, you can’t just go to the folder PIP installed it to and compile it there. That’d be too easy of course! (This took me a long time to figure out)… If you try to do that, it’ll fail with cryptic errors.  The reason is that the file path is too long.

To overcome this, you have to perform the following steps:

  1. Clone pyinstaller’s source to a folder in your C drive, then
  2. Rebuild the bootloaders,
  3. Install pyinstaller with pip,
  4. Overwrite your pip installation with your newly built bootloader files…

Duh, obvious right?! (omgwtfwhyisthisalwayssohard!?)

Details for the steps are below:

Step 1: Open a powershell as admin and go to C:\\

cd c:\\

Download the source of pyinstaller. This will create its own folder for pyinstall:
>pre>git clone https://github.com/pyinstaller/pyinstaller

Step 2: Cd into the bootloader’s build folder:

cd .\\PyInstaller\\bootloader\\

Then run the script to reinstall. If you get an error telling you there’s no such thing as ./waf then you are in the wrong folder.

python ./waf all

Step 3: Once this is done, you can go back to vscode or wherever and install pyinstaller from pip

pip install pyinstaller

Step 4: Then navigate to python’s site packages, and copy the newly built bootloaders into the appropriate place. NOTE: PyInstaller is case sensitive here, so be careful. We’ll first make a backup of the original bootloader folder:

 mv C:\\Users\\ALaptop\\AppData\\Roaming\\Python\\Python38\\site-packages\\PyInstaller C:\\Users\\ALaptop\\AppData\\Roaming\\Python\\Python38\\site-packages\\PyInstaller.bak

Then copy in our freshly compiled bootloader:

mv C:\\PyInstaller\\bootloader C:\\Users\\ALaptop\\AppData\\Roaming\\Python\\Python38\\site-packages\\PyInstaller

Note: Now to use pysimplegui-exemaker, you must edit your computer’s %PATH environment variable to locate pyinstall. Click the windows key on the keyboard, type “environment” and click to open the first suggestion. This window will pop up that has a button towards the bottom named “Environment Variables” that you must click. In the top window pane, find “path” or “PATH” and doubleclick that line to edit it. You’ll now be able to enter a new value. You want to add the path to where pip installs its scripts.  In my case, I pasted in the following:

 C:\\Users\\ALaptop\\AppData\\Roaming\\Python\\Python38\\Scripts

Then test it out:

python -m pysimplegui-exemaker.pysimplegui-exemaker #<--If you use pysimplegui's exemaker (NOT a typo, must exactly this way) OR

pyinstall --onefile ./myProgram.py #<-- If you just use pyinstaller directly

Tada! No more virus warnings!!!!

 

Check out my other post about how to replace the default python icon in your executable files.

Using VSCode and Mkdocs to make a static website

For FabAcademy, students have to post each of their assignments on a website. The website must be a static HTML site hosted on Gitlab (though this method works for github too).  There are a lot of ways to generate static sites, but maybe one of the simplest seems to be to use mkdocs.

I made a video tutorial showing the steps required to get up and running quickly with this method. Just a couple of tweaks required here… In the video I apparently missed a couple of steps. If you have trouble getting python from within VScode’s terminal to work, you need to set the path in VSCode’s settings as well as the PATH environment variable in windows.  VSCode’s tutorial on installing python and selecting the proper python interpreter shares some steps to help you debug if you have this issue.

The second thing you might have to do is set your name and email in git. Try this:

git config --global user.email "<your email here>"
git config --global user.name "<your name here>"

The other steps in the video should get you going.

I will say I did thi