It’s been over a year since I created the first version of my Ambilight clone, but I finally found the time to return to it and add some new features. I was originally planning on adding colour correction (to correct for the purple wall behind my TV) and output delay (to get perfect synchronisation with the TV), but I got a bit carried away and have added a whole raft of new stuff.
The specification now looks like this:
- Direct HDMI input at up to 1920×1080 60Hz
- Drive up to 8 strips of LEDs at full frame rate
- Up to 512 LEDs per strip for a total of 4096 LEDs
- Each LED can take it’s colour from any one of 256 arbitrary screen areas
- Each LED can use one of 16 colour correction matrices
- Each LED can use one of 8 sets of gamma correction tables (separate tables for each R, G, B channel)
- Output can be delayed up to 8 frames with microsecond steps for precise synchronisation with TV
- Configurable temporal smoothing
- Up to 64 configurations can be stored in flash memory
- Automatic letterbox/pillarbox detection which can trigger loading of different configurations from flash memory
- Configurable via serial console with a greatly extended command set
The rest of this post describes the implementation details, but with all the changes it’s got quite long, so alternatively you can go straight to the demo videos at the bottom of the page, or the code over on github.
Hardware
HDMI Light v2 Hardware
The most notable change is a move from the Papilio One 250K to the Papilio Pro. I’ve had to make this move as the previous version wasn’t far from filling the Spartan-3E FPGA on the old board. That combined with the Spartan-3E’s lack of any DSP slices (dedicated multiplier / accumulator hardware) and the fact that I’d used almost all of the block RAMs meant I had very little chance of being able to squeeze in the colour correction, let alone any of the other new features.
The Papilio Pro has a Spartan-6 LX9 FPGA, which among other things increases the number of logic cells from 5,500 to 9,100, increases the number of block RAMs from 12 (24KB) to 32 (64KB), and adds 16 DSP48 slices.
The HDMI receiver board remains essentially the same, I still haven’t fixed the output level shifter or the power jack bugs from the previous version, they don’t stop anything working so they’re low priority. The Papilio Pro board also uses a switching regulator that can deliver more power then the linear regulator used on the Papilio One, so the power jack on the receiver board is even less necessary than it was on the previous version.
The move to the Papilio Pro has also fixed the clock net bug. By luck the move means that the LLC clock output of the ADV7611 HDMI Receiver IC is now connected to an FPGA pin that’s connected to a clock net, which is nice!
The only actual change to the receiver board is a change to the resistor values for the DDC and CEC pull-ups. The ones I was using previously were 1,000 times too strong and prevented the EDID information from being retrieved by the HDMI source.
FPGA Design Changes
HDMI Light V2 Block Diagram
The following sections give more detail about the implementation of the new features.
Colour Correction
Problem Wall
White
Green
Skin Tone
In theory, correcting the colour should be fairly straight forward. The purple paint is absorbing more of some of the colour components of the light that’s shone on it than others, stopping those components from being reflected and seen. If the colour sent to the LEDs either boosts these components by an amount proportional to the extra absorption, or attenuates the other components by the opposite amount then the perceived colour should be corrected.
Of course my dark purple wall is a fairly extreme case and no matter what I do I’m never going to make it look bright white, but as the images above show, I can at least make white lights appear to be more grey than pink. In case it’s not obvious, in each of the images the LEDs on the left are colour corrected and the LEDs on the right are not. There’s a piece of cardboard stopping the light from each side interfering with the other.
This correction could be done by simply multiplying each of the colour channels with a constant correction factor to scale the output of each channel. However, by expanding from a set of three constants to a matrix of 16, it’s possible to go from simple scaling to a whole world of possible transformations.
An RGB colour can be considered to be a point in a 3D space, where instead of having the X, Y and Z axes representing physical space, we instead have R, G and B. By doing this, any transformation that could be applied to a 3D vector can also be applied to a colour, simply by multiplying the vector that represents the colour by a transformation matrix.
Possible transformations include:
- Brightness: implemented using a scaling transformation
- Contrast: implemented using a combination of scaling and translation
- Hue: implemented by applying several rotations and a shear
- Saturation: implemented using a shear
It’s also possible for a single matrix to do all of these at once, simply multiplying together the matrices for each individual transformation results in a single matrix that performs all of the transformations.
To implement this I’ve added a new colourTransformer component, which is used by the resultDistributor to modify the colours before they’re sent to the outputs. The transformation matrices are stored in a single block RAM and the resultDistributor uses the output mapping table to set the upper address bits of the RAM to select the appropriate matrix for each LED.
The colour transformation is performed over six clock cycles using three multipliers and three accumulators, plus a counter and a few multiplexers to control the inputs to each stage of the calculation. The six cycles perform the following steps:
- The first cycle just selects the first row of the matrix, nothing else can be done until the coefficients are available on the output of the RAM.
- The second cycle then multiplies the R, G and B values by the first set of coefficients and requests the second row from the RAM
- The third cycle multiplies the R, G and B values by the second set of coefficients and adds the result to the previous result
- The fourth cycle multiplies the third set of coefficients
- the fifth cycle multiplies the fourth and final set of coefficients
- The sixth cycle clamps the accumulated result to the range 0-255 and signals completion
Gamma Correction
Gamma: 1.0 (top) vs. 2.2 (bottom)
The perceived brightness of LEDs is not linear with the RGB values that are sent to them. They tend to get bright very quickly through the lower values, but then there’s very little noticeable change at the higher end. For example the difference between 0 and 15 is quite noticeable, but the difference between 240 and 255 is practically invisible.
Gamma correction fixes this by defining a curve that is used to modify the output so that a linear change in the value calculated for an LED also appears to be a linear change to our perception.
I’ve implemented gamma correction using lookup tables, which made it the simplest of the new features to implement. There are three 2KB block RAMs, one for each of the R, G and B channels. Each of these block RAMs contains 8 separate tables, each of which contains 256 8-bit values.
To perform the gamma correction the incoming R, G and B values are used to set the lower 8 address bits on each of the block RAMs (the upper address bits are used to select which of the 8 tables is used). Then on the next rising edge of the clock the resulting R, G and B values are available on the data out of the RAMs.
The tables can be populated using the equation:
table[i] = ((i / 255) ** gamma) * 255
Output Delay
Delay 2 Frames
No Delay
To allow for this the output can now be delayed by up to 8 frames, and there is further fine tuning that can adjust the output delay in steps of 1 micro-second.
The two images above show the output from a pair of photo-transistors, the yellow trace is the LEDs and the blue trace is the TV. The first image shows how the LEDs are switching on too early when there is no delay and the second image shows that the synchronisation is almost perfect with a delay of two frames (video during tests is 50Hz).
The delay is implemented by a new resultDelay component that is inserted between the lightAverager and the resultDistributor. This component can delay the trigger signal that tells the resultDistributor to start sending data out to the LEDs, and it can also present different results to the resultDistributor than those from the current frame.
The whole-frame delay is achieved by remembering the last 8 sets of results from the screen area averaging. Each set of results from the averaging consists of 256 24-bit colours, so a 32-bit wide by 2048 row block RAM is used (consuming four 2KB block RAM primitives). This RAM is used as a ring buffer, the first set of results go into the first 256 rows, the second set of results into the second, and so on, wrapping around to the beginning again after the eighth.
With the last eight sets of results available in the ring buffer it’s then simply a matter of using the frame delay count to offset the read location, so that the resultDistributor is reading results from the RAM at write_pointer minus frame_delay.
One important detail is that when the frame delay is set to zero, the signal that starts the resultDistributor can’t be set until the new results have been copied into the delay RAM.
The micro-second fine tuning is implemented with a simple counter. When the copying of the latest results into the delay RAM has been completed the counter starts counting down, when it reaches zero the start signal is sent to the resultDistributor.
It turns out that there’s really no benefit to such a fine level of adjustment, but now that I’ve done it there’s not much point ripping it out again. As long as the output is synchronised within less than one frame then it appears as good as perfect, which is lucky really if you consider how long it can take to update the LEDs. If one of the strips has a full compliment of 512 LEDs then it takes over half a frame’s worth of time to clock out the serial data to the last LED.
Temporal Smoothing
Delay 0, Smooth 0.75
Delay 1, Smooth 0.65
This smoothing applies a sort of rolling average to the calculated colour of each screen area. It implements the following equations:
R = (Rprevious * X) + (Rcurrent * (1 - X)) G = (Gprevious * X) + (Gcurrent * (1 - X)) B = (Bprevious * X) + (Bcurrent * (1 - X))
Where X is the desired level of smoothing, between 0.000 and 1.000. This means that a value of 0.000 gives no smoothing, with the output being 100% from the current frame. A value of 0.500 has the output set from 50% of the previous colour and 50% of the current.
The calculation is performed with three multipliers and three accumulators (one for each colour channel) over two clock cycles. During the first cycle the address is set on the delay RAM to lookup the previous result and the current result’s R, G and B values are multiplied with the coefficient. During the second cycle the previous result is available from the delay RAM and its R, G and B values are multiplied with the coefficient and added to the results of the first cycle.
The calculations are performed using 9.9 unsigned fixed point numbers, which means that the smoothing can be configured in steps of 0.002.
To access the delay RAM the temporal smoothing is borrowing the read port that’s normally used by the resultDistributor, but that’s OK as the resultDistributor can’t get the start signal until it’s done.
Configuration In Flash
Flash Memory Map
In this version it’s possible to use the spare capacity in the flash memory on the Papilio board to hold 64 separate configurations and to switch configuration with a single serial command.
Initial population of these configurations in the flash memory happens at build time. A set of text config files (which reside in the config directory) are parsed and rendered into a binary form and appended to the FPGA bit file. The bit file can then be written to flash as normal.
Transferring data between flash and the CPU’s RAM and between flash and the ambilight configuration RAMs and registers is handled by a new flashDmaController component. The transfers are initiated by the CPU by setting a flash address,
a RAM address, a length and a direction. The CPU is then halted while the flashDmaController copies the data, after which the CPU continues.
Halting the CPU is kind of cheating, but it’s a lot easier than double clocking the RAM and providing simultaneous access for both the CPU and DMA controller.
Format Detection
Letter-boxed Content
Now that it’s possible to store multiple configurations in flash memory, it’s possible to have one configuration for full screen content, one for letter-boxed content and one for pillar-boxed content. All that’s needed is a way to switch betweeen them.
I’ve added a new formatDetector component that examines the incoming video data to find the active area. Like the scaler it takes its input from the hscale4 component rather than the raw incoming video so that it’s got four clock cycles available per pixel.
It’s essentially just a collection of counters:
- A counter that measures the number of pixels in a line.
- A counter that measures the number of lines.
- A counter that measures the number of lines at the top of the screen before the first line that has an average brightness above a threshold (height of top black bar).
- A counter that measures the number of lines at the bottom of the screen after the last line that has an average brightness above a threshold. (height of bottom black bar)
- A counter that measures the minimum width from the start of a line before the brightness goes above a threshold (width of left black bar).
- A counter that measures the minumum width after the last pixel on a line that has a brightness above a threshold (width of right black bar).
At the end of each frame, if any of the counters has a different value when compared to the last frame then a signal is raised which causes the CPU running the firmware to receive an interrupt.
The remainder of the format detection is then done in the firmware. Given the size of the screen and the size of the black bars around the active area of the screen, the firmware works as follows:
- If the active area has invaded the black bars of the current format or the resolution has changed then reset to the default 16:9 ratio
- If the active area of the picture fills the screen horizontally and the active area is centered vertically and it approximately matches a 2.4:1 ratio then switch to 2.40:1 format.
- If the active area of the picture fills the screen vertically and the active area is centered horizontally and it approximately matches a 1.33:1 ratio then switch to 1.33:1 format.
There is a format table stored in flash memory and whenever the format changes the firmware walks the table until it finds an entry that matches the resolution and aspect ratio of the active area. If it finds a matching entry then it loads the configuration specified by that entry from flash.
Output Mapping
The final change within the FPGA is a completely rewritten resultDistributor component, which is needed to tie together all of the new features.
The previous version was configured by a single table with 256 entries. That table contained one entry for each of the 256 LEDs that could be driven, defining the rectangular area of the screen that should be averaged to get the LED’s colour and the index of the output that the result should be sent to.
The output index was relayed through together with the colour resulting from averaging the screen area, so that the result distributor component could iterate through the 256 results and control a demultiplexer to direct each result to the appropriate output.
The new version decouples the LEDs from the screen areas, which allows for there to be more LEDs than screen areas, and also allows the LEDs to have individually configurable colour correction.
The decoupling is achieved by having a mapping table associated with each of the 8 outputs. Each of these tables contains 512 entries, and each entry defines the screen area to take the LEDs colour from, which colour correction matrix to use and which set of gamma correction tables to use.
The new result distributor component iterates through each of the 512 output mapping entries from each of the 8 output tables, interleaving the tables so that it first does row 0 from table 0, then row 0 from table 1, etc. It uses each entry in the output mapping tables to set the addresses for the result RAM, colour correction matrix RAM and gamma correction table RAM. It then takes the colour from the result RAM and passes it to the colour transformer component and waits for the result be be available. It then uses the transformed colour to address the gamma correction table before finally taking the output of the gamma correction table and passing it to the WS2811 driver for the relevant output.
There are now 8 separate WS2811 drivers, and unlike the previous version they’re all driven in parallel. Clocking out the 24 bits of serial data for each LED takes a long time, so the result distributor starts the driver for output 0 and then uses the time it takes to clock out the data to perform the lookups and transformations for output 1, then output 2, etc. Even by the time it’s started the last output driver, the first one still hasn’t finished outputting the first LEDs data, so it does the lookups for the second LED on channel 0 and then waits for the driver to be idle, which allows it to start clocking out the next LEDs data immediately after the first one finishes, with no gaps.
Firmware Changes
Serial Console
Supporting all of the new features has required an almost complete rewrite and it’s also meant that I’ve had to quadruple the size of the memory used to store the code, from the 4KB used in the previous version to 16KB in this version.
I’ve also changed the way that the firmware accesses the configuration RAMs and registers. In the previous version the access was via a set of 8 bit ports, and every byte written to the configuration required writing to one port to set the high bits of the config address, writing to a second port to set the low bits of the address, and then finally writing the data to a third port. In this new version the configuration RAMs and registers are mapped directly into the upper 32KB of the AVR’s memory map, so writing a byte to the configuration is as simple as writing to RAM. As you can probably imagine, this greatly simplifies a lot of the code.
I won’t cover the code here, it can be found in the firmware directory in the github repository and should be fairly self explanatory.
The command set is also documented in the repository in the file docs/serial-commands.md
Demo Videos
The set up for the following videos has 20 LEDs between each of the three shelves on the left (at the back near the wall), another 20 between each of the three shelves on the right, 39 LEDs on each side of the TV illuminating the front of the shelves, and 2 rows of 66 LEDs on top of the TV for a grand total of 330 LEDs. They draw almost 10 Amps for full white and light up the whole room.
All LEDs are using a gamma setting of 1.6, they all have colour saturation reduced to 0.7, and the LEDs on the top of the TV are combating the purple wall with brightness set to 0.7 for red, 1.1 for green and 0.6 for blue.
It’s been incredibly difficult getting a camera to produce a representative image, the colours in these videos aren’t very accurate and the bright bands to the sides of the TV aren’t anywhere near as visible in real life, instead there’s a much smoother gradient.
Files
Everything needed to recreate this can be found in the github repository:
- The README file contains build instructions
- The board directory contains the Eagle CAD files for the HDMI receiver board
- The fpga directory contains the VHDL code and Xilinx ISE project files
- The firmware directory contains the C code for the firmware
Pingback: FPGA Ambilight Clone Packs a Ton of Features | Hackaday
apologies to anyone that’s been trying to comment, seems I broke something… all working again now.
Do you have a bill of materials or a cost breakdown for your configuration featured in the videos? I’m trying to asses the feasibility of attempting something like this.
I’ve never added it up myself, but a commenter on my post for the original version came up with a figure of $208 (http://hacks.esar.org.uk/hdmi-light/#comment-1836).
Accounting for a Papilio Pro instead of a Papilio One will bring that nearer to $250.
Then add 2x 4m of 60 LED/m strips at about $40 each gets to $330.
Add a 5V 8A PSU for $15 is $345.
It looks like there’s someone working on a BOM over in the hackaday comments which may be worth keeping an eye on:
http://hackaday.com/2014/12/16/fpga-ambilight-clone-packs-a-ton-of-features/comment-page-1/#comment-2258104
Circuit/Board amendments are complete a few things fixed (DC socket, TXB0108 and TPD12S520). BOM is nearly complete. Please can you provide the part number for WE-CBF_0805 (L2-L7), it comes in a variety of impedances.
Also, the 28.6363MHz crystal, I can’t find one with a 47pF load capacitance. Do you have a part number, or can I get away with a 28.6363 with a load cap around 20pF?
Thanks
Excellent!
The ferrite beads I used were MURATA BLM21PG220SH1D 22ohm @ 100MHz:
http://uk.farnell.com/murata/blm21pg220sh1d/ferrite-bead-0-009ohm-6a-0805/dp/1515658
The crystal I used was TXC 9C-28.63636MEEJ-T:
http://uk.farnell.com/txc/9c-28-63636meej-t/xtal-28-63636mhz-18pf-smd-hc-49s/dp/1842321?ost=1842321
It’s quite possible that I had the same problem finding a part as you and then never updated the schematic, shame caps don’t have markings otherwise I could check the board and make sure I’m using 18pF caps. I do have a very vague recollection of the crystal failing to oscillate and then changing the caps, though that could have been a different circuit.
btw, I’ve just been making some of the same changes, as Jack over at the Gadget Factory (home of the Papilio) is interested in doing a run of the boards.
I’ve also rearranged the pins so that the ADV7611 clock pin is on a global clock input on both the Papilio One and Papilio Pro. You may want to make the same change to maintain compatibility with future releases. I’ve just got a couple more checks to make before I’m done with it.
What did you replace the TXB0108 with? I’ve just dropped in a 74HCT541. If you’ve done something different, let me know, I might change to match.
Let me know when you’re done and we can compare notes and merge the changes together. Also, I’m sure Jack will find the BOM really helpful, especially where you’re finding errors in the schematic!
I went with the 74HCT245 as it’s reported to work well with NeoPixels. If you reckon the 74HCT541 is better, I’ll switch.
Please can you provide me the exact details of the clock pin change, i.e. which pin to which pin.
Drop me an email, and I’ll send you through the updated BOM / pcb layout/schematic.
list of changes so far:
Fixed DC Power Connector, Assigned missing parts, replaced ESD Protection with TPD12S520, fixed eagle warnings and issues that didn’t meet OSHParks design rules. Added 0 ohm jumped to TPD12S520 LVL, for replacement with resistor as per data sheet. Change TXB0108 to 74HCT245. Put indication a few legends on board at I/O locations and added cap/led orientation indicators. Used low ESR caps on regulator outputs. Removed silkscreen from overlap on bias.
Looks like we’ve gone for near enough the same solution for the level translation. As far as I can tell the only real difference between the ‘245 and ‘541 is that the ‘541 is one way only.
The rearrangement of the pins moves the LLC output from Papilio pin B9 to pin B13, it also moves the remaining pins up to fill the space and rearranges the four spare pins to be in a half-wing arrangement with power, which is hopefully more useful. The changes have also required shuffling around some of the power traces and bypass caps, so rather than try to explain it all I’ve uploaded my work-in-progress version:
http://hacks.esar.org.uk/board-v1.1-work-in-progress.tgz
I hadn’t spotted the extra resistor on the TPD12S520 low voltage supply, and your other changes sound good too.
Regarding the low ESR caps, you’re probably right to change them. On this board the regulators don’t oscillate, but when I used them on a different board with the same two caps on the input and output they did. Possibly the whole collection of caps on the output side combines to make it work.
I gave up looking for caps that match the datasheet requirements though, seemed to be near impossible, at least from Farnell’s stock.
I’ll be interested to see your version of the board. Depending how many changes there are it might be easier for me to redo the pin rearranging on your board.
To get my email address, replace the first dot in this site’s domain with an at, so hacks @ esar …
Hey Esar,
I saw this project title on hackaday and thought wow, I’d love to see an ambilight clone on the Papilio. Then I opened the article and saw it was on the Papilio! Sweeeeet! What an awesome job with this project. 🙂
I’d really love to get one of these boards built for myself and I think that the easiest way to do that is to get a quote from my manufacturer’s and see if we can build a batch of these boards just like any other Papilio MegaWing. Can you drop me an email at support@gadgetfactory.net so we can talk about possibly doing a batch of these boards?
Congrats on such an amazing project. 🙂
Jack Gassett
Papilio Designer
I would be very interested in buying if anything would be available. I could also design a 3D-Printable enclosure.
Cheers
AB
Pingback: Gadget Factory | HDMI Light V2
Any change to by this PCB (maybe with soldered SMD parts?)
Very great project!
by = buy
Pingback: Gadget Factory | HDMI Splitter is also a Decrypter
I have a miniSpartan6+ Lx9, two hdmi ports built in for I/O. How would i go about adapting your design to my board? Awesome job btw. Also which led strips you use?
I’ve just had a quick look at the miniSpartan6+ and see that the HDMI ports are directly connected to the FPGA. Unfortunately the Spartan6 FPGAs aren’t quite fast enough to do 1080p input on their own. That would leave you with two choices:
1) If you want 1080p, then adapt the layout of my HDMI receiver board to fit the miniSpartan6+. Then it would just be a case of adjusting the pin layout in the .UCF file and the existing VHDL code should just work.
2) If you don’t need 1080p, you could add HDMI decoding functionality in the FPGA and connect the output of that to the existing RGB data bus within the FPGA code.
For an example of HDMI input decoding within the FPGA, see this guys work:
http://hamsterworks.co.nz/mediawiki/index.php/HDMI_Input
Hi Esar,
If i program papilio pro, serial port will function without your pcb? I can’t get it working. Guessed parameters from sources be 115200,n,1 but it show offline in minicom.
I’ve just checked on mine, after taking the receiver board off the serial prompt no longer appears. I suspect this is because it’s getting stuck sending I2C commands to initialise the board. This should be fixable by commenting out the #define AUTO_INITIALIZATION at line 37 of main.c, though I haven’t had time to try it.
Another thing to check is that you have both hardware and software flow control turned off in the minicom settings.
You have correctly guessed the other settings: 115200,8,n,1
Hi Esar,
Finished assembling your board. Connected to papilio pro without any checks (not have lab power supply) and its go at first :). Console working, hot plug of hdmi working, amazing. This is my first smd soldering job. Not to hard if have soldering station and magnifying glasses. :). Today is to late for other checks but im think this thing is working. 😉
Congratulations, that’s great news!
Good luck with the rest of your testing and config.
Hi Esar,
It works, thank you for your job. Now i need to work on config. First two diodes light near white all time and 129 and 130 does not light, i guess this is config mismatch. My strip is 5m 30/meter i must buy more ;).
Hi Esar,
i’m just trying to compile the python files with python 3.4, since i don’t have 2.7.
Therefore i changed some things parts in the pyton files because it gave me some errors. For example i changed sys.stdout.write(str(width >> 8)) instead of chr.
But now i get some error i can’t fix anymore. In the python file bitmerge are some print like
print(“Section ‘%c’ (size %6d) ‘%s'” % (section, length, desc))
i put parentheses since it changed from 2.7 to 3.4. But there i get an error i don’t understand:
ValueError: invalid literal for int() with base 10: b’a’
I’m quite new with python and i think that the change of chr to str might have caused the problem, but i don’t now what else to do.
Thank you very much for your help,
Fabian
ok i fixed it by using python 2.7, but it would still be interesting to know what i would have had to change.
Thank you,
Fabian
Glad you were able to get up and running. I don’t have python 3 installed on any of my machines at the moment, but if I can find a spare few minutes at the weekend then I’ll try and see what the issue is.
Hi Esar,
Thank you for your quick reply.
I got it working, but still one smaller issue:
The GPIO(0) doesn’t go high, i accidentally found when i pull the Papilio pin A0 high i get a “power on” in the serial console and then it works. But if i remove the pullup, i get an “power off” and it stops working. Semms that it has something to do with that line:
GPIO(0) <= PORTD(0) when DDRD(0) = '1' else 'Z';
Or what else could cause the problem?
When the pullup is attached, and i move something over my cloned monitor i see nice and smooth color change as it is supposed to be, so i think the HDMI interface is working fine.
Thank you very much,
Fabian
Ah, yes, I should have had that disabled by default.
The power on/off state (not really, it just pretends the screen is all black) can be controlled via remote control commands over HDMI CEC, or it can be controlled by adding a physical pull-up and switch. Because the later requires extra hardware I should have disabled it by default.
I’ve just pushed a fix to github.
Hi Esar,
thank you! I have one question left, im trying to build my own config for my screen now but it seems i can’t load it, or i don’t know how to change it.
If i type GM 0, on your screenshot i see you get an answer, i only get some G [1B][1DM [1B][1D
For example i changed the output 0 in config 0 that in the first WS2811 output red and blue are swapped, but it doesn’t work.
When uploading the bitfile, onfig0 is loaded by default, right?
Thank you
Fabian
Changing the config should just be a case of editing the conf files in config/config_0 and then running make in the top level directory. When you run make the following should happen:
1) Each modified .conf file should be used to regenerate the respective .bin file via the appropriate python script
2) All of the .bin files in each of the config_x directories should be glued together to produce config/config_x/merged.bin
3) All of those merged.bin files from each of the config_x directories should be glued together to produce config/merged.bin
4) The config/merged.bin file should be appended to hdmilight-preconfig.bit to produce hdmilight.bit in the top level directory.
So… if you modify config/config_0/output_0.conf and run make in the top level directory you should see the last modified time change for:
* config/config_0/output_0.bin
* config/config_0/merged.bin
* config/merged.bin
* hdmilight.bit
Reflashing the Papilio Pro should then have the changes active.
Initially the active configuration is not from these config files, it’s a hardcoded config applied by the various Rst functions (cmdRstArea, cmdRstOutput, etc) in the firmware. The config in flash is made active by one of two methods:
1) Issuing a GM command via the serial console
2) Via the picture changing to match an entry in config/format.conf
Method 2 will happen whenever the picture changes format, this can be disabled by issuing a DF command.
The output you describe when issuing “GM 0” is similar to what I had in the past when I had a bug in the way I was halting the AVR CPU during the transfer of the config from flash to RAM, but that should have been fixed long ago.
When you issue the “GM 0” command you should first see “loading config…” printed to the serial console. Does this happen?
Next the CPU is halted and the transfer takes place, then once the CPU is restarted is should continue and print “done.”
I’ll make sure mine is running the latest version from github over the weekend and see if I get the same issue. If I don’t have the problem then I’ll upload my bit file so you can try that. Then we’ll know if the problem is with the sythesis/compiler/other tool versions.
I’ve just done a clean build of the current github version and the GM command is working on my board. If you’re still having problems with it, can you try this bit file:
http://hacks.esar.org.uk/hdmilight-v2-a1c923d57582ec2350b274e4815648252fc5b7bd.bit
That’s the exact file I just flashed my board with. I then did this:
> GA 0
0: 0 0 0 8 3
> GM 0
loading config...
done.
> GA 0
0: 0 7 34 34 3
>
That’s getting the start up values for area 0, loading config 0 and then getting the values for area 0 again.
Hi Esar,
Thank you very much for your effort, i just uploaded your file but i still have the same issue:
GA 0
G [1B][1DA [1B][1D
> GM 0
G [1B][1DM [1B][1D
> GA 0
G [1B][1DA [1B][1D
>
Thats what i get back from my serial console.
Seems like i have a problem with my board and i don’t now what kind of. It is exactly the same i get back when i use my bitfile. I use the Papilio Pro with the XC6SLX9 Tq144BIV. I also sometimes get some errors uploading the bitfile, maybe this is associated with the serial problem. But somehow the serialconsole works.
If i type GS i get back:
G [1DS [1D
status: 0
>
I used Termite and Arduino serial console.
Thank you very much,
Fabian
It think this is a problem with the serial terminals that you are using. I’ve just tried the Arduino serial monitor and have the same issue as you:
G [1DA [1D
> G [1DM [1D
> G [1DA [1D
> G [1DS [1D
status: 3
>
The [1D and similar are ANSI control sequences for cursor movements, which the Arduino serial monitor doesn’t understand.
It’s strange that the output of the GA and GM commands is completely missing, while the output of the GS command is visible, but as they both work on the same board when using minicom instead of the Arduino serial monitor, it must be an artifact of the terminal.
Hi Esar,
You were right, my serial console was the problem. Thank you for that. No everything works and i can load the configs but if i use for example config_0 output_0: area 0 is led 0,
area 1 is led 1 and so on, up to 150 for example, only led 0 – 26 light up. The rest stays dark or in its previous state.
The funny thing is, if i set a new Output Map via serial, i can also light up 27 and more.
If i use the GO command, it also says that LEDs with higher indexes than 26 have no mapping.
For me it looks like something is not loaded properly.
Thank you very much!
Fabian
I want to start in the bottom left of the screen go up (34 LEDs) and then to the right (76 LEDs) and finally back down (34 LEDs again).
If i type R all 124 Leds light up and are controlled by the video.
I also used the C program for config and area generation, same problem. I have no idea where this comes from.
That’s a bit strange, it sounds like it’s either not producing the right data from the config files before merging into the bit file, or possibly that the merge is failing.
Can you send me the contents of your config directory, including the built merged.bin files?
I’ll take a look and see if I can spot where it’s gone wrong.
My email address is the same as this site’s domain, but with the first dot replaced with an @, so hacks at esar …
Hi Esar.
One more question. In addition to papilio pro I have papilio one 500k and assembled hdmi receiver from hdmilight v2. My question is: can I connect hdmi receiver from v2 to papilio one 500k whithout changes in design?
Unfortunately not, there aren’t quite enough resources in the FPGA on the 500k for the design to fit. If did fit then it would just have been a case of changing the target device in the ISE project settings and modifying fpga/hdmilight.ucf to change the pin mapping, but if you did that you’d find that after synthesis the device utilisation report would be over 100%.
Version 1 would fit, but it’s missing a lot of features and contains a few bugs that I’ve never got around to fixing. That would just need the target device changing and the .ucf file updated to match up the pins.
Hi Esar,
Sorted out how to create config mapping leds to areas, thanks to ConfigGenerate, rewrite some because i need counter-clockwise from bottom right corner of screen. This working perfectly. But i messed up about creating configuration for color correction and gamma. Can you explain how to do in clolor_x.conf, gamma_x.conf files and serial console? For example i have yellow wall – how modify files and how to do from serial console?
The colour correction is controlled by a matrix that each set of RGB values is multiplied with. The colour_x.conf files also have options to set brightness, hue and saturation, but the end result is still a single matrix.
You can view the matrix via the serial console using the GC command. To retrieve all four rows of colour matrix 0:
> GC 0 0-3
0 0: 1.000 0.000 0.000 0.000
0 1: 0.000 1.000 0.000 0.000
0 2: 0.000 0.000 1.000 0.000
0 3: 0.000 0.000 0.000 0.000
The first row of the matrix defines the red channel, the second the green channel, the third the blue channel, the forth is a constant that I’ll ignore for now.
The output above is the identity matrix, which is the default do-nothing matrix. It says that the output for the red channel gets it’s value by adding 1 times the red, 0 times the green and 0 times the blue channel. Similarly the green output is from 0 times the red channel, 1 times the green and 0 times the blue.
The simplest modification to the matrix is to change the brightness of each channel, simply by changing the 1.000 to a lower value to reduce the brightness of that channel or a higher value to increase it. This is all I needed to do for my purple wall. Purple is a mixture of red and blue so my wall is reflecting more red and blue than green. Therefore I reduced the brightness of the red and blue channels and boosted the green channel. My matrix then looked like this:
Rin Gin Bin Const
Rout: 0.700 0.000 0.000 0.000
Gout: 0.000 1.100 0.000 0.000
Bout: 0.000 0.000 0.600 0.000
Cout: 0.000 0.000 0.000 0.000
That reduces the red brightness to 70%, increases the green to 110% and decreases the blue to 60%.
If your wall is yellow then it is reflecting more red and green and less blue, so you’ll want to reduce the brightness of the red and green channels and boost the blue. You can play around with the values via the serial console SC command.
To reduce the red brightness to 80% in matrix 0:
SC 0 0 0.8 0 0 0
To reduce the green to 80%:
SC 0 1 0 0.8 0 0
To boost the blue to 110%:
SC 0 2 0 0 1.1 0
The changes will be instant so you can play around with the values to find the best effect and then update the conf files to make it permanent.
For the gamma you probably can just use the same 1.6 value that I’ve used. It’s not really practical to change it via the serial console as although the config file only contains a single number, that number is used to produce a table of 256 values. This table is then used to adjust the output, so for example if the output for the red channel has a value of 25 it reads the 25th entry in the red table and outputs the value it finds there.
You can view the table via the console using the GG command, the following will retrieve all 256 values for the red channel of gamma table set 0:
GG 0 0 0-255
You can also set them via the SG command but it would take forever. Set element 0 of the red channel of table set 0 to 128:
SG 0 0 0 128
An almost but not quite complete reference to the serial commands can be found here:
https://github.com/esar/hdmilight-v2/blob/master/docs/serial-commands.md
All of this colour stuff is a hard subject to describe, but hopefully the above makes some sense.
Hi Esar,
Is possible to force format? For example when watching film width subtitles on bottom bar, format is not recognized correctly. Some stations have logo on black areas and same effect. Maybe button or something else. Four pins of fpga is not connected, maybe we can use this for force format?
Automatic config switching when the format changes can be disabled via the console using the DF command, you can then manually load a specific config via the GM command and it will stay on that config.
It would certainly be possible to use some of the spare pins to do this too. If you know your way around AVR C code then the pins are accessible as the lower 4 bits of PORTD and if you look in firmware/main.c around line 104 you’ll see how one of the pins is being used to signal power on/off. Otherwise I’ll add it to my TODO list, but it will probably be a while before I can do it.
I don’t realizing those pins connected to AVR I invented, connect those pins to formatDetector module and write simple counter from 0 to 3 for button presses (0 automatic, 1 fullscreen, etc) then force parameters for format on: xSize, xPreActive, xPostActive, ySize, yPreActive, yPostActive. But your way maybe simpler. I look at this and share success or fail :).
Hello, Esar!
I am very interested in your project, it is truly unique. Can I buy this product for personal use? There are several instances of the device. Mail me please.
sorry, no, I’ve only ever made one.
I loved your board!
Please… Can I use it with Raspberry Pi (A, B or B+)?
If “yes”, how can I command your board with the Rpi?
Thank you so much!
Amazing project.
Best regards.
Marco / Brazil.
Which bit do you want to use with the Raspberry Pi? Do you mean the HDMI receiver board or the combined receiver board and Papilio Pro?
If you mean the HDMI receiver board, then no, that can’t be used with a Raspberry Pi.
If you mean the whole thing, then yes, it can be used with anything that can talk to a USB serial device. You can simply plug it into one of the Raspberry Pi’s USB ports and then use something like minicom to send it commands to configure it.
Hi Esar,
Can you get me some idea how can I switch to a larger AVR core?. Now Makefile have defined atmega16. I found some code for IRReceiver (which i want to use for format switching) for AVR, but in atmega16 this does not fit with your code together.
The atmega16 in the Makefile is really only there so that the compiler generates machine code that is compatible with the AVR that is implemented in the FPGA. I think the tools are quite happy to ignore the 16KB program size limit.
To increase the size of the program memory the FPGA code needs to be updated to connect some more block RAMs to the AVR. Luckily there are 5 unused 2KB block RAMs remaining in the FPGA. Due to the way the AVR core is designed (it stores odd and even addresses in different RAMs) they must be used in pairs, which means 4 of the 5 can be used, which would extend the program memory from 16KB to 24KB. Hopefully that will be enough for the IR receiver code to fit.
This needs to be done in fpga/avr/prog_mem.vhd, I’ll try to make this change over the weekend.
This is now done and pushed to github, the AVR now has 24KB of program memory instead of the 16KB that it had before.
I’ve only given it a very quick test but it seems to be working ok. The space utilisation report printed out at the end of the build still thinks that there is only 16KB available but you should be ok to push it to 150% utilisation.
Hello, i saw your project, i am fascinated by it and have a question because i am working on my own project with ADV7611. How do you know the timing diagram for the HSYNC VSYNC and DE. the manual is pretty big and saw some diagrams but they are not very clear to me. Do you only use the DE to determine and count the pixel data?
You’re correct, I’m using DE to find the actual image data.
Originally I just started a new line when I saw HSYNC and started a new frame when I saw VSYNC, and this mostly worked because my line buffer was big enough that the extra data fitted without overflow. It just meant that the image data was offset slightly, but that wasn’t much of an issue for driving my lights.
Eventually I changed to use DE to make things more accurate. It is only asserted during the actual image, so with a 1920 pixel wide image, DE will be asserted for exactly 1920 pixels.
But if you only use DE, how do you know you started at the start of the frame?
By the way very cool project you got!
Essentially I have two counters, one for x and one for y.
The x counter is really simple, it’s clocked by LLC from the ADV and is reset by an inverted version of DE, so when DE goes low at the end of the line, the x counter is reset to zero and held reset until DE goes high again. From this the x counter counts exactly from the first pixel of the real image data to the last.
The y counter is reset to 0 when VSYNC goes high, so at the start of the frame. Now this on its own would cause the count to start before the real image data, but I don’t use HSYNC to increment it, instead I increment it on the falling edge of DE. So I reset to 0 on VSYNC, then it stays at 0 until DE has a falling edge at the end of the first valid line of image data. It is then incremented every time DE falls at the end of each line.
If your familiar with VHDL you can see this implemented here (it’s a bit ugly, but it works 🙂 ):
https://github.com/esar/hdmilight-v2/blob/master/fpga/scaler.vhd
Good luck with your project. BTW, what is it?
Ahh thanks i get it now!
pretty smart! to use the DE in combination with the VSYNC instead of HSYNC.
At the moment i am trying to get HDMI signal, and send it over ethernet to my computer. bit to much probably for the arty but i wanna see what i can make of it. and doesn’t have to be 1080p60hz it’s is also for the learning
Hee,
I have almost everything working for the project only i ssee u also tried it with the RPI, now if i try to force my RPI on 480p the ADV7611 doesn’t see the image, when i let it do auto resolution , i get an image. have u any experience with this?
It’s been a long time since I had it connected to an RPI, so I can’t remember what resolution I was testing it with.
To help get an idea of whether the problem is with the ADV7611 not accepting the signal, or whether it’s a configuration problem, can you post the output of running the GF command (Get Format) at the serial console?
EDIT: Ah, just notice which thread you’ve posted in, if you’re not running the hdmilight firmware, you obviously won’t be able to run that command. If that is the case then I would look into whether you are getting a clock signal from the ADV in the 480p mode. It might be that the ADV is not recognising the mode and needs a few register values tweaking to get it to recognise it. If there is a clock then it would be interesting to see if the frequency is approximately what would be expected for 480p or if it’s nearer the frequency used for free run mode when no signal is present.
Pingback: My Ambilight, My Obsession, My Curse | The Brain of Bryan
Great Project!
Are you using the papilio’s dedicated RAM?
I am thinking of building a nice all-in-one PCB for this.
No, it’s only using the FPGAs internal block RAMs. The memory requirements are very low as it process one line of video at a time. In fact roughly half of the RAM is dedicated the the AVR processor’s program memory rather than the video processing.
Let me know if you do produce an all in one board, that would be very cool.
Could you please share hdmilight.bit? I understand it is customized, but I’d like to bypass dealing with Xilinx, setting up ISE and getting the licence.
I’m absolute newbie in FPGA programming, but your project is awesome
Im half the way to create the board (single board (4cmx6cm) combine 2 boards (papilio and HDMI receiver))
after digging around, I found that Makefile has to command on Ubuntu to make it works
but can I just upload the HDMIlighttop.bit that has been created by the ISE, and then using terminal to config number of LED after, because I fund the config about the gamma and some color correction is not much important
I still dont understand the output port that can drive 8 led strips. so those strips is continuous, ( last led number 62 is in the 1st strip then if I connect the second strip, the first LED of second strip is led number 63? and so on….
thank you for giving us an awesome project!!!
finally the board has done. I dont know if any mistake when combining 2 boards
http://2.pik.vn/2018059da10d-53ea-443a-a472-e18d89072b87.png
Hi!
What an awesome build!
I’ve built a Hyperion setup myself, but not really satisfied in how the video is taken from a HDMI splitter, then converted to composite signal and then fed into a usb capture card. It feels like way too many steps, together with loss of quality and delay. This looks waaay cleaner.
In your first version you used a PC with HDMI and DVI, is your current setup the same? Would an AV-receiver with double hdmi output be a good fit for this?
How difficult would you say the building of the HDMI receiver is? Given that i’ve only got programming knowledge of development boards, no real experience in designing and building boards.
I read in the comments that the miniSpartan6 wouldn’t be powerful enough to process 1080p. Is that true for both Spartan6 LX9 and LX25?
/Erik
Yeah, I still use the PC with HDMI going to the TV and mirrored to the DVI for the HDMI Light. While I got everything working with an HDMI splitter, it introduced a slight delay when switching sources, and as I watch everything from the PC there was no point in keeping it in the loop. A receiver that can mirror the output to two HDMI outputs would be perfect.
The main factor in building the HDMI receiver board is soldering ability. If you’re comfortable soldering larger stuff, know how to cleanup accidental bridges and have a magnifying glass then it’s not actually that hard, but it probably wouldn’t be a good place to start as a first soldering project. The hardest parts to solder are the main chip and the HDMI connector, they’ve both got pins that are only 0.5mm apart.
The comments regarding the miniSpartan were about getting the FPGA to take in the HDMI signal directly without the separate receiver board. For that the whole family of Spartan6’s are too underpowered. Reading the raw HDMI signals directly requires some very fast processing that Spartan’s just can’t do. Modifying the VHDL code to take the signals in directly would also be non-trivial.
Thanks for the quick reply.
Might be a good time to give the bottlehead crack diy kit a go then, to improve my soldering skills.
There was some talk about Gadget Factory running a batch of the board. Do you know if they did? Or if it’s still planned?
As far as I know Gadget Factory never did a run. We talked about it a bit, but then they went quiet. I’m guessing they probably decided that there wouldn’t be enough demand to make it worthwhile.
I’m currently using WS2801 leds. Would it be possible to modify the code to drive these? And would it be difficult?
I wasn’t familiar with the WS2801’s, but I just had a quick look at the datasheet, and they look like they’re easy enough to use. So theoretically, yes, it should be possible.
It would require reducing the number of output from 8 to 4 to account for needing a clock line for each one. Then it would mainly be a case of swapping in a replacement for this file in the VHDL code:
https://github.com/esar/hdmilight-v2/blob/master/fpga/ws2811Driver.vhd
Sounds promising. I will let you know if I get around to giving the build a try and if I (hopefully) succeed 🙂
Hi.
Great project. I would lile to make one .i ordered the pcb and it will be delivered after a week i believe.
What are the things to co sider if i am making this.
Regards
Ben
Hi Ben, good luck with the build. If you’re soldering by hand rather than using an oven or hotplate, then making the connection to the big ground pad under the ADV7611 can be a bit of a pain, but other than that I don’t remember having many problems.
Let me know if you have any problems and I’ll try and help if I can.
Hi,
this is an awesome project! I am trying to figure out how hard it is to extract multichannel audio from an hdmi signal in a way that allows me to run some dsp filters on them, preferably on a low cost device like a Raspberry Pi. Would it be hard to adapt your hdmi board to something that outputs the sound signal via i2c? Are you aware of any drivers that can deal with this under linux? Thanks!
JC
Hi, the ADV7611 chip on the board outputs the audio as I2S. It should be possible to add a couple of bodge wires to break out these signals and send them to the Raspberry Pi’s GPIO pins. I know STM32 microcontrollers can use their SPI peripherals to receive the I2S stream, but I’m not sure whether the Raspberry Pi has anything similar.
Hello,
Great project – i like to build it, but i have one question, because it is not clear to me where are the Output Pins for the LED Strips – and how are wire them to the board.
Hi, the LED output is the pin header next to the HDMI connector. There are 8 pairs of data+ground to drive up to 8 separate LED strips.
Hi Esar,
first of all, thank you for your project and effort. It’s a great project and works pretty well.
However, I got one little problem and could need an advice from you.
Short intro:
– x: Pixels 66
-y: Pixels 36
-Top Pixels are working.
-Left/Right only half strip. Bottom not working.
When using the GF command from serial console the output is: 1916×540.
So my guess is, that it only recognises 50% of the height during format detection and lit only half of the vertical stripe.
Do you know about this behavior and could provide an idea for a solution.
Thank you in advance.
Denis
Hi, I’ve not seen that before. It does seem very suspicious that 540 is half the 1080 that you would expect.
What device is the source of the HDMI signal? Could it be sending an interlaced signal, so 1080i instead of 1080p?
I don’t think I’ve ever tried it with an interlaced signal, but it would make sense that it would send half height frames.
If it is interlaced then you should just need to adjust the numbers for the area definitions
Hi esar,
It is a wonderful project, I am very intrested , and I want to make a single board for this, but i am not familiar with FPGA, now i plan to design the schmatic, when it flished, could you please help me to review it , and whether there is a much better FPGA IC for the project, as I found you use a FPAG kit board maybe just for DIY’s convenience. Waiting for your reply, Thanks.
Hi, your message on github prompted me to check the messages here 🙂
I have limited spare time, but I’ll try to help review if I can.
Regarding the FPGA, the Spartan 6 part is really the only suitable one. There are two things that make it that way:
1) The spartan 6 has a relatively large number of block RAM resources and I’m using lots of them.
2) It’s one of very few remaining FPGA parts that have legs, and is therefore relatively easy to hand solder.
The spartan 6 is the only part that I’ve found that meets both requirements. I did try looking for alternatives recently, but came up blank. I was hoping that one of the Lattice parts that work with the open source tool chain would have worked, but no luck, nowhere near enough block RAMs.
I have also been working on an all in one board, though not with a built-in splitter. It might be useful to you, if only to see how I’ve done some of the layout between the FPGA and ADV7611. One important thing here is to try to keep the lengths of the individual data signals as similar as possible. I’ve pushed it up to github here: https://github.com/esar/hdmilight-v3
Bear in mind that that board has not been tested and may not work. I’m currently waiting for boards and stencils to arrive from China. Hopefully it will be good. This is the second revision, the first one mostly worked after I added 6 or 7 bodge wires to fix the bugs.
OK, Actually I am in China, and familiar with PCB design and PCBA manufacture etc., if you want to make a mass production I can help.
Thanks for the offer, but I don’t have any plans to mass produce. I just do this for fun. It must be handy being in China when you need boards. For me 90% of the time and 90% of the cost is shipping.
HI,
i thought of starting with v2 and found your comment.
for sure i will start with version 3 only 🙂
Let me know if you have any problems. I’ve done a bit more work over the last few weeks and v3 should now be at the point where all the features supported by v2 are now working ok. Though it’s still had relatively little testing.
Hi all
I have designed the single board for the V2 version, everything work perfectly^-^,
details as the link https://github.com/xfce/bi4wmsproject/wiki
Hey guys.
This is nearly exactly what I’m looking for!
Don’t suppose you have been working on a v3 for 4K resolutions?
Sorry, no, I’m still using a 1080 TV, so haven’t looked into reworking it for higher resolution
Make the production version include a HDMI spliter, which can bypass a 4K/1080p resolution to TV and downscale the 4K to 1080p, so now the hdmi-light can also handle 4K video, and you can find this on tindie
https://www.tindie.com/products/bi4wms/fpga-ambilight-controller/