Friday, May 16, 2014

Automating FPGA builds to allow for coding across all platforms

There have recently been a lot of development boards appearing on the market containing a combination of an ARM CPU and an FPGA, allowing for both software and hardware development on the same board, which allows for many interesting projects and hacks. Recently, the Novena in particular has caught my eye, I highly recommend you take a look at their page to see some of the features!

However, each of these boards has a pretty glaring problem in my eyes, and that is the lack of support for any fpga bitstream generation tools on ARM, meaning you would have to use another PC to develop for the FPGA and then transfer files around. All is not lost however, there are ways around this problem!

Solution:

Use that x86-based server you have that's spends 90% of it's time doing nothing and make it do all the work for you! Allowing you to code for your fpga using any device as long as it has access to the internet, git support, and some kind of text editor.

Here's how I did it:

I'll probably post each step of this in more detail over the next few weeks, but for now here's a quick overview of how my system works.

First, you'll need a server to house your projects git repository, with enough space to also hold an installation of the Xilinx tools (this requires around 20GB or so), I currently use and recommend the basic Linode plan (referral link). In order to install the Xilinx tools I just gzipped up my /opt/Xilinx folder and scp'd it over, then extracted it into the same directory. I also had to install the webpack license file into ~/.Xilinx/Xilinx.lic for each user that was going to be using the tools.

You will then need to set up a git server if you haven't already, this isn't too hard to do, instructions can be found here.

You'll also need to set up your server to be able to send you emails (receiving isn't necessary), there are various guides on different methods of doing this (and your server likely already has this set up), so I'll leave this up to you for now.

Now for the good stuff, making git build the code and send back the .bit file in an email (or at least the logs if there is an error). Luckily git makes this very simple! The git developers were smart enough to include some useful hooks, one of which happens to run after each push to the server is completed!

In order to make use of this, you first need to create an empty git repository on your server:

(as the git user)
mkdir repo_name.git
cd repo_name.git
git --bare init

Now navigate to the .hooks folder in this repository, and create a post_receive file. This will contain a script that will be run after each push to the server is completed. Edit the file and paste in these contents (modify them where necessary for your own project and preferences):

post_receive:
#!/bin/bash

MAKE_OUTPUT=output.bit #change this for your project
TEMP_DIR=/home/git/temp_git_stuff # make sure this directory exists and is able to be used by whatever user is controlling your git repos
EMAIL=your@email.com # make sure to put your own email ;)


CHECK_COMMIT=`git log -1 | grep BUILD` # you can make it whatever you want, I prefer BUILD though.

if [[ -n $CHECK_COMMIT ]]; then
        GIT_WORK_TREE=$TEMP_DIR git checkout -f
        cd $TEMP_DIR
        source xilinx_settings.sh # see below, this sets up my xilinx environment, replace with settings32 or 64.sh if you'd prefer
        make > make.log
        curr_time=`date +"%F %T"`
        if [[ -e $MAKE_OUTPUT ]]; then
                cat make.log | mutt $EMAIL -s "Build Success: $curr_time" -a $MAKE_OUTPUT -a make.log # you can remove the initial cat so that you will have an email with no body text
        else
                cat make.log | mutt $EMAIL -s "Build Failed: $curr_time" -a make.log
        fi
        rm -r $TEMP_DIR/* # clean-up, optional
fi

Save that file and make it executable (I use chmod +x post_receive). If you use that file exactly you'll also want to use my xilinx_settings.sh file. You can source the normal Xilinx settings{32,64}.sh file if you'd prefer, mine just does the same thing but works in zsh as well:

xilnx_settings.sh:
#!/bin/bash

# Fixed settings file for Xilinx ISE Development tools
# May need to update XILINX_DIR and SETTINGS_FILE based upon your own setup
# This probably wont work for all shells, and it isn't exactly an elegant 
# solution, but it does make things work on my zsh setup, so I'm happy.
# Note that the commented code near the bottom might help make things a 
# little nicer/cleaner, though I was unable to get the for loop just right
# for zsh to accept it, I'll leave it there in case other wish to give it a go.

XILINX_DIR="/opt/Xilinx/14.7/ISE_DS"
SETTINGS_FILE=.settings64.sh

. "$XILINX_DIR/common/$SETTINGS_FILE" "$XILINX_DIR/common"
. "$XILINX_DIR/EDK/$SETTINGS_FILE" "$XILINX_DIR/EDK"
. "$XILINX_DIR/PlanAhead/$SETTINGS_FILE" "$XILINX_DIR/PlanAhead"
. "$XILINX_DIR/ISE/$SETTINGS_FILE" "$XILINX_DIR/ISE"

Now all that you need is a project with a Makefile! (you can see an example of one I have created here for the Basys 2 board from digilent).

Unfortunately I haven't yet found a "catch all" Makefile for an FPGA, so I just created my own that works for my particular setup. I've posted it below and you're welcome to use parts of it if you want, though I recommend first doing a full build within ise, and then reading through the cmd_log that is generated and comparing to my makefile below to make sure that all the commands match up.

link to makefile

If you use this makefile, I recommend a directory structure used like in this project here: https://github.com/emusan/graph_radius. The ise project is based out of the ise directory, to keep all of its files tucked away nicely, and a separate build folder is created by make to store temporary build files, this can be easily deleted with a make clean and ignored using the gitignore file. Note also that if you use this makefile you will have to edit the ise/{project}.xst file to remove the top two "set" lines and to make the -ifn parameter set to ../ise/{project}.prj. If you don't then there will be a {project}.prj can not be found error (note that because of the ways that the directories are set up you wont have to change this again if you want to use ISE :D).

This is all still pretty hacked together, as I just got it all working over the past few days, but so far I haven't run into any serious problems (aside from the usual issues when trying to figure out each part). In the future I'll probably try and improve upon it some, but for now I don't plan on changing it too much.

If you've got any suggestions let me know! There's the comments on here and my email is around various places. I've also been hanging around on the oftc irc on channel #kosagi, discussing the new Novena laptop!

Thursday, May 15, 2014

I'm back?

It has been quite some time since I've posted here. School always seems to catch up with me when I least expect it, and then one thing leads to another and suddenly I'm completely off the blogging track, and once you're off getting back on can be quite the task.

I've considered just starting up a new blog to try and make myself blog more (because somehow that works out in my head), but I figure I'll just stick with this one for now, I may move in the future and if I do I'll be sure to make a post here (assuming anyone reads this still). There are definitely some very nice looking blogging platforms out now, with less of the crap that things like this and wordpress bring along which is tempting, but at the same time I do like to be able to take in comments, etc.

Aside from all that, quite a bit has changed since I last made a post here! I've since moved from using microcontrollers as often to making use of FPGAs more and more, I think I'm rather addicted to them. That means a lot more posts (assuming there will be any) will be focused more on FPGAs, FPGA software, and different ways of using both for various projects.

I do have a few posts planned out in my head right now, in particular a series on how to automate the build process for FPGA bitfiles. The goal for this is to have my server host the git repository, then whenever I make a push to it of my updated code, it will pull from itself, synthesize and build the bitfile, then make that available for me to download easily. Perhaps down the line I will also have it send me any errors or warnings that were encountered during the process. Having an automated server like this allows me to use any kind of PC, tablet, or phone to write vhdl code, independent of that systems CPU architecture, operating system, etc. All it will require is a text editor, git, and internet access.

That's about all for now though. I'll be writing up some more posts tonight and over the next few days probably, but I'm going to try setting them up to release on a schedule, so that my random bursts of content will appear to be more evenly spread out.

Thanks for reading!

Monday, February 18, 2013

STM32F4 Character LCD


Finally another post about electronics! (and yeah, haven't been blogging as much as I'd like, life has been busy busy busy)

Code can be found here.

I recently just completed a project (as a precursor to another project) to interface a 4 line by 20 character HD44780 LCD with my STM32F4 Discovery(link to digikey, ST site is being updated right now). The project is fairly simple, but still decently educational (particularly if you have little experience with strings in C). Overall it took about three weekends to complete, though would have taken much much less had I not hit a wall partway through due to a typo in the code, but it all works now! (I may make some minor changes to it at some point, but it is feature complete as of now, with the exception of custom characters which will come in a week or so).

The library itself is not particularly well optimized, in order to achieve the slow communication speed required by the LCD I used delays in the code, rather than a timer on the chip, which results in a huge number of lost cycles. I plan on doing another, more optimized version of this at a later date using timers, though I will leave this intact as it is far easier to understand and adapt to other projects/controllers. The timings used in the delays will also probably be reduced as I find more time to experiment with them, but for now they are still pretty quick (see the second half of the video below for the real speeds).

The video was taken on my phone so I know quality isn't great,
watch it on youtube though at 480p to get the best out of it (not much I know)

It should be simple enough to adapt the library to your needs, though I do plan on adding some comments to the code and could write a tutorial if people want. The code is designed to work well no matter how many lines or characters per line your LCD has, the only thing that would need additional work is if you wanted to use 4-bit communication, which would require a bit of rewriting for sending data (in "nibbles" as opposed to all at once), as well as the initialization, but the rest should work fine. I am always willing to answer any questions that people have as well!

If you want to use the same LCD that I used, you can get it from seeed studio here, though they do take awhile to ship. This LCD doesn't use an HD44780 driver, but the one it uses is pin compatible. The library should be pretty easy to adapt to a 2-line display as well, most of the code is there and all of it should adapt well to other numbers of lines and characters per line.

As for the electronics side, I hooked it up like this (taken from pinout.txt in the repository):

LCD     Function      STM32F4
1       Ground        -------
2       VCC           -------
3       Contrast      -(pot)- [may use GPIOD 6 at a later date]
4       RS            GPIOD 7
5       R/W           GPIOD 8
6       Clock(enable) GPIOD 9
7       Bit 0         GPIOE 4
8       Bit 1         GPIOE 5
9       Bit 2         GPIOE 6
10      Bit 3         GPIOE 7
11      Bit 4         GPIOE 8
12      Bit 5         GPIOE 9
13      Bit 6         GPIOE10
14      Bit 7         GPIOE11
15      Backlight(+)  GPIOD10 [potentially]
16      Backlight(-)  -------

GPIOE4-11 were chosen for the data as they are the only contiguous(code-wise) bank of 8 pins I could find on the F4 Discovery board that are not attached to an external peripheral. Choosing them like this allows for a nifty bit of code for writing the data, seen here:

GPIOE->ODR=((GPIOE->ODR&0xF00F) | (data << 4));

This sets the ODR (Output Data Register) on GPIOE to either the original state for non-data pins (so as to not interrupt other functions of the controller when using the LCD), or the state for our data (which is shifted 4 bits("pins") up so as to not disturb GPIOE0-3).

Please feel free to let me know if this helps you in your project!

Enjoy!

(and a few more detailed pictures:)

 A very pretty display module from seeed

An absolute mess of wires, but it works!

Monday, January 21, 2013

2013! Also, been awhile, updates, etc. (will be posting more often too!)

So, been a very long time since my last post, too long. Not long after the last post I had a break from school and then it was finals and I had zero free time. I had meant to make a post or two over my winter break from classes, but just kept putting it off. Anyway, I very much enjoyed the break, but here I am back at school and ready to do some real electronics work and start posting again!

As I had mentioned before I got a Nexus 7, and I planned on creating a chroot environment to install Arch Linux (my favorite distro by far) on. I tried a number of things since then and ended up finding Linux Deploy, an open source program for Android that does everything I had wanted to do and more! (it supports multiple popular distros and has some great options for configuring them out of the box). I can't recommend this program enough to anyone that wants a genuine Linux experience on their Android device.

I'm currently using it mostly over ssh (through either better terminal emulator pro [BTEP] or zshaolin, both great apps for Linux lovers, with zshaolin essentially being a zsh version of BTEP), though I've tried a bit of VNC stuff with it as well, I'm currently trying to find the best Android VNC client (currently Pocket Cloud is looking decent, though has some annoyances and free version limitations). Anyway, this tablet is shaping up to be one of my favorite tech purchases ever, I use it as my phone (more on that later), an internet tablet, and now as a mini PC (particularly great when connected to an external keyboard).

In other news, I have just moved my domain (somestupidwebsite.com) to a new registrar, and will likely be switching hosting companies soon as well (not that doing so changes anything here). The new registrar is gandi.net, and so far I'm very pleased with their service! Once I switch the VPS host I might even write a sort of "review" type post for both gandi and the new VPS host that I'll use (I'm currently looking into Linode, they seem to be one of the better reviewed options and will let me use Arch!).

As well, I can now promise (to some degree) new posts on a bit more of a structured schedule, I'm going to be writing another one or two over the next day or so and then setting them up on a queue to post each week or so.

Enjoy!

(oh and a little peak at my current project -> linkey)

Thursday, November 8, 2012

Nexus 7

Well, been a little while since I've updated this, school seems to keep me pretty busy. However, I will have some free time this weekend during which I hope to work on (and maybe complete) a project or two. Which should mean more posts here!

As for the title of the post, I did just recently get a nice new toy/productivity device, a Google Nexus 7 (writing this post on it now in fact). I've already rooted it, and I plan on doing a few more fun/interesting things with it soon.

Hopefully I will be able to write up some posts and perhaps guides for some of what I do (plenty of rooting guides out there as is, but few if any custom chroot ones, I will be setting mine up with Arch to match my laptop). Once I have that properly set up I should be able to edit code, and even compile while on the go, amongst other things, which should translate into more posts here.

I'm also planning on being more active on Google+ now, so if you enjoy this blog and wanna chat/see smaller posts from me check me out there (I believe it's linked to this account, I will post a link later though if needed).

More to come soon!

Saturday, September 29, 2012

STM32F4 Discovery Hello World (with blinking lights!)

What better way to start off a new blog than with a hello world!? Last week a lovely little STM32F4 discovery board arrived at my door from digikey and this weekend I finally got around to hacking on it. First step was to set up a toolchain (I'm on Linux and using the summon arm toolchain as of now), this was pretty simple to do. Then I downloaded the development libraries from ST, which can be found here, along with tons of other great info about the microcontroller included on the Discovery board.

The next step was to set up a nice, simple, and adaptable template project that I could use as a basis for future developments. Parts of this were adapted from a template I set up for the STM32F1 that I had used previously, along with start-up and linking code from ST's examples. In the future I may personalize more of it, but for now it suits me just fine. The code in there now blinks the four LED's in order, clockwise from green to orange, then red and blue. It's a very simple example, but it's not hard to adapt this little code to do a whole lot more! All the code for this project can be found here.

After creating the code and compiling it, I used a jtag adapter to program the chip itself. The Discovery board comes with an ST-Link programming adapter, but the Linux support with that is somewhat minimal from what I've seen, and I already had my jtag adapter and environment set up how I liked it.

I may write a tutorial soon describing the process of setting up something like this, as well as one on how to use the template "properly". I will also be posting here regularly with updates on various projects made with this microcontroller, including some LCD interfacing soon hopefully!

Here is a little video of the demo running(sorry for the shaky camera, just using my cell phone right now):


And some nice higher resolution photos of the board itself:







This shows my JTAG debugger(yellow PCB) as well

Anyway, that's all for now~ Enjoy!

Monday, September 17, 2012