Reworking MIDI RX, how midi sent to Grid is processed

Grid
1

Each control element has a dedicated LED, with options for custom RGB colors, intensity change and LED animations. These are all powerful features to provide feedback on interactions with Grid and controlled parameters. But what happens, when the parameter states are not set by turning a knob on a controller, but it’s reported from the controlled software or hardware back to Grid?

Well you go to the system events, select the midi rx event and try to figure out how to map changes to specific control elements. It’s not an intuitive process, I think we could do better.

Let’s discuss what kind of bidirectional communication is the most useful for you. Do you know an example where this works right? What would you except from Grid, when you send such data to it? Should it work with standard MIDI, sysex or else?

We already started to work on support for scaled value pickup, so this might be interesting not only for encoders, but potmeters as well.

4 Likes
2

I’ve been experimenting with setting up my new grid controllers (4 x en16),
as a 8x8 grid of RGB Led encoders, that reflect a RGB matrix on my pc, using bidirectional communication.

I use the grid to set values with the encoders in relative mode, so when I turn an encoder, midi is sent to my software. Then a value is read from my matrix, on either the red, green or blue channel, then it’s adjusted by the encoder increment or decrement, and the final value is sent back to the grid controller, updating the LED in question.

I can choose which channel I’m interacting with either by paging, or simply inside the software.

further more, when I recall a preset, or saved state for the matrix on the software side, all the RGB values are sent out to the controller, in order to reflect the updated state of the matrix (64 x 3 values, index/RGB).

So far I can make it work just fine, but I have a few questions about ways to optimize, and improve the code and functionality.

My experiments so far include; sending different midi commands, in order to address the 3 independent RGB values,

control-change for red.
note-on for green.
after-touch for blue.
then using a lookup object inside grid editor, we can map the midi.cmd’s to indices.

(example)
cmd → color-channel
176 → 0 (control-change)
144 → 1 (note-on)
160 → 2 (after-touch)

Alternatively I’ve been midi.ch to differentiate between the RGB color variables, which eliminates the need for the lookup object.
red = cc on channel 1
green = cc on Ch 2
blue = cc on Ch 3

then I setup a self variable in the Init-page called rgb = {0, 0, 0}
so I can later set the RGB values on the individual grid elements like this:
element[i].rgb[color-channel] = midi.param2*2
then finally update the led color:
led_color(i, 2, element[i].rgb[0], element[i].rgb[1], element[i].rgb[2])

However when I set all the values when recalling on the pc,
all the 64 x 3 values (cells x color-channels), have to be limited to 10ms intervals.
If I attempt to send midi out to the grid controller any faster,
I seem to encounter errors, either stuck Led’s, or entire controllers blinking,
so I have to limit the speed at which I update the controller which isn’t ideal.
64 x 3 x 10ms ~= 2 seconds per update.

I’ve tried limiting the frequency at which the Led’s update,
by introducing if statements in the midi RX actions,
so that led_color will only change when the blue channel is updated,
Since I send out red, green and blue for every cell,
there is no need to update until all three have been received.

another potential bottleneck is the fact that I can’t seem to address individual grid controllers when I want to.
If I sent a midi message to grid, all the devices receive the same message,
hence I need to filter messages on the midi RX actions, using module_position_x() and module_position_y(), and more if statements,
that ensure that the device Led’s only update when the midi.param1 is within the range of the controller. which may be inefficient.

(example)
pos = (module_position_x()*16)+((module_position_y()+1)//8)
if param1>=pos and param1<(pos+16) then
i = param1 - pos;
element[i].argb[ch] = param2 << 1
if ch == 2 then
led_color(i, 2, element[i].argb[0], element[i].argb[1], element[i].argb[2])

I also tried changing to code to using bitwise operators instead of multiplication and division, where possible,
but it doesn’t seem to have an effect on efficiency, or the allowed minimal update refresh-rate.

(example)
pos = (module_position_x()*16)+((module_position_y()+1)//8)
pos = (module_position_x()<<4)+((module_position_y()+1)>>3)

I’ve worked with other DAW controllers, that are able to update almost instantly, through regular USB 2/3 ports (no USB-c),
devices like the icon Qcon daw controllers, or novation launchpads, can update super fast,
so I’m wondering if there are any obvious ways to speed up the communication from pc to grid devices?
or if I’m simply misunderstanding how to leverage the grid studio correctly.

I’d like to test sending midi sysEx messages to the grid devices midi RX page,
but I can’t find any documentation on how to parse sysEx on the grid device side,
only info on sending sysEx from Grid and out.
Maybe sysEx would be a faster way to communicate, since I could format a single long message,
containing all 64 x 3 values wrapped within a single sysEx message.
Instead of sending 64 x 3 separate messages out vie USB.

  • sorry for the long rant, but I’d rather provide as many details as possible,
    so hopefully it’s easier to understand my experience with bidirectional communication on grid so far.

I look forward to any suggestions on how to optimize my setup further,
and perhaps more options will be available in the future, if midi RX is getting reworked.

Either way, I’m loving the controller so far!

  • Svend/Gussi
3

Hi!

Thank you for the well explained post.

This is a really complex issue as the real problem itself lies indeed in the way Grid modules work as a system when using MIDI RX.
As you summarized, all messages sent to Grid are sent to all individual modules, where they have to be sorted/filtered with conditional statements under the MIDIRX event.

So right now, the limiting factor for MIDI RX is the following as I understand it is the following.

When receiving MIDI, each message is forwarded to the other modules to make sure no message is lost. So the hosted module sends these out on side PCBs to conneected Grid modules and waits for the messages to arrive on the other modules.
But now, if the hosted module receives a new message during this time from either the computer or any other module as it’s waiting will cause the errors you experienced.

Now normally this shouldn’t happen, because the serial communication between modules should be faster than the messages received from the computer, but in multimodule setups, this becomes a very complicated routing issue and all communication slows down.

Currently the hosted Grid has no idea if the messages properly arrived to all modules until they really get there and in a circular setup, like yours with 8x8 grid matrices messages will bounce in all directions slowing this process down further.

So the MIDI RX rework should deal with exactly this behavior. We’re attempting to make the way Grids spread messages between each other smarter, faster and prevent loops like this from happening.

This is my thoughts on the matter. Keep in mind that I’m not the engineer dealing with the protocol itself but this is my understanding of the issue.

(Small note on your earlier question, there is no way to parse SysEx on Grid yet, but we’ll make this happen, since many of you asked for it. SysEx could be a way to solve this, I agree. Reducing the number of messages you send is the key in the current form of MIDI RX.)

Thank you for sharing your experiences, we’ll be able to use this as an example setup that should work perfectly, but it just doesn’t.

4

Thank you for the reply,

just as a quick side note,
I did also test the 4 x en16 grid devices side by side, in a 16x4 setup.
And the issues with whole devices flashing (usually blue on all Leds),
where still present, and as far as I remember, limiting the update frequency on the midi messages from my pc to grid at ~10ms was still necessary for stability.

For now I’ll keep testing,
and keep you updated if I find a better solution.

5

Hello!
I just bought a few Grid Elements to replace my cheap midi controllers (Launchcontrol XL + more)

Most controllers on the market let you define LED colour by a midi note velocity message. It is usally the same note number that is used send notes on that button/led.
So button 0 and LEd0 use the same Note number (0) to send and receive midi. Velocity will determine LED colour/intensity

I apreciate the fast response of your customer service but to set all of this up is impossible as a non programmer :frowning: This kind of beahaviour should be part of the standart templates! Literally every 100€ ableton controller on the market can do this out of the box.
I really love the ethos and the design of your products but this brings me to the edge of sending these units back…

1 Like
6

I think we discussed this on discord lately: Discord

Handling midi rx and doing even “simple things” with it, require coding knowledge. We will work on this, thanks for sharing your frustration.

Not sure if this helps to point in the right direction: More about: Receiving MIDI | Intech Studio Documentation

Happy to give a rundown with profile / preset for your exact use case, but for that maybe we should open a new topic.

7

Yes, we are on discord with this. I’m also happy to continue the discussion there, thanks for your help!

8

MIDI RX is now considered top priority starting next year.

We will start with improving the UX and configurability first. When we have larger user base who can confidently use it, we can fix upcoming stability concerns. We found that it’s hard to figure out where stability falls short, when getting to setup is so cumbersome.

9

Yes, the midi feedback issues need to be resolved. I’m seeing the same with Grid modules acting up when receiving midi feedback continuously

10

I’ve spent a month now with the grid controllers and with the help of my friend (Main developer for Soundbrenner) we’ve come up with grid profiles that I can test in practical situations.
In these tests, even with midi rx messages limited to 10ms the leds on the grid are not updated reliably.
I tested it both with Axoloti hosting grid and Ableton and documented it in these 2 videos:

You can see that even with only 2 modules connected the errors occur. So the stabilty of this is realy worse than I had expected :frowning:

These are the profiles for the 2 bu16:
grid-editor://?config-link=4QAb0NYvtcEmO8ZS3ug3
grid-editor://?config-link=RNkcMkIL9Cvn8pUBPFSc

and this is the ableton test session:

If it’s not stable at all, I don’t see any motivation in using this feature so I don’t know if this logic will work out this way. Would’nt more users trying to use a broken feature only result in more complaints?

Also I think the broad user base would need a “standart” implementation of midi feedback based on how Novation, Akai, Livid, Korg and all these companies do it.
Personally I’m a bit more tech savy, but I think for a “standart” user the general confguration with the editor is not accessable. Even simple things like changing a midi note are not straight forward.

11

hi

I can only support user19: as long as the Backend is not stable, nobody will ever use the feature no matter how nice the UX is. So i would definitely focus on the Backend first. As soon as the Backend is reliable, we can upload a few Templates how to use it until your finish with the UX Redesign to help out other Users. But it all depends on a working Backend!
As long as the whole Grid crashes and the Parameters sendet from the Grid are starting to flicker as soon as the Grid receives to many Midi Messages, it is not usable as Midi Controller. At least not with Encoders, which need working feedback. An this is REALLY disappointing for a Controller in this Price-Segment!

So absolute top Prio to solve in my Point of View (an should not even have to be discussed for such a midi Controller!):

  • Reliable Midi-RX without crashing or loosing Midi Messages!!

And speaking about priority: even if the Topic according to Kristof is Top Priority right now, there was no Progress at all MIDI RX implementation status #72 in almost a Month…

12

Hi,

I’ve been busy reworking the NVM bulk operations that cause dropouts in editor. This was due to NVM read-write operations (profile load/clear, page chage etc.) causing unacceptable jitter in packet timings. This is one of that key problems that prevented ESP32 based modules from performing up to the same reliability as the old SAMD51 based variants.

Now that this is done, the next focus is to improve the efficiency of network layer routing on the ESP32 based variants.

ESP32 based Grid boards have an RP2040 co-processor onboard that act as a Quad UART to SPI interface chip. This currently receives serial port packets from all four directions and sends them to the main ESP32 chip for processing via SPI. The main processor then decides what to do with the message and sends a response back via the SPI interface to the co-processor. One it is received the co-processor sends the response out in all four directions.

The SPI transfers between the two processors cause added latency even if the packet in question is not mean for that Grid module at all. This development effort aims to upgrade the co-prcessor firmware to take care of network layer routing independently, and only send relevant messages to the main ESP processor.

Interesting facts:

  • Latency in networking is very problematic, because the longer a packet takes to reach its destination the longer it is required to take up RAM memory in one or more of the networking components. This can cause buffer overflow then resulting to packet loss!

  • Older D51 based Grid variants currently have lower latency, because they do not use interfacing co-processor, since they have enough serial ports to handle the four-way traffic required for Grid communication. The above mentioned reworking of responsibilities between ESP32 main processor and RP2040 co-processor will result in similar latency figures between the two architectures while reducing load on the main processor.

Hope this gives you some insight into the delicate nature of this issue. We are dedicated to delivering a reliable solution for MIDI RX while also maintaining compatibility with older D51 based Grid modules.

13

Hi Suku

First of all: thanks for the fast and detailed explanation - really appreciated!

Don’t get me wrong: i now it is no easy Task! I have some knowledge in small-Bandwith Mesh-Network Protocols on embedded Systems and i now that there are always some side-effects which popup and need a lot of time to fix. But you also need to understand our side: we spend a lot of Money on a Controller which now is not working as expected. There was no “Warning: midiRX is not fully working right now” or “Product is still in development” Information at all. And to be honest: Encoders without working midiRX are almost useless as they don’t update and have no visual feedback.

I still believe in the Product as the Hardware, Formfactor, Modularity and the Lua Capability have a lot of Potential. That’s also what kept me from sending them back, even if i have a e1 next to them. But i’m also really hoping that we get working midiRX soon.

Thx & BR

14

I must concur, I’d expect midi feedback was a given for such a device as well.
I’m still keeping my Grids and appreciating them a lot, but the feature is essential and I hope its sorted out soon too

2 Likes
15

Everyone, with the release of v1.2.49 Editor, we can start testing internally the firmware improvements meant to improve the networking capabilities of Grid.

2 Likes
16

Any update for standar MIDI feedback?

17

Or a layout example i can test with encoders?

18

You can test with the EN16, using the very basic preset called ‘en16 midirx’ found in the Profile Cloud. It feeds back to the default configuration of the EN16.

It might be already good enough depending on your use case. There were improvements to multi-module stability in firmware version 1.2.32, but I’m would say we’re not done with improvements yet.

I still experience slowdown/lag when using data-heavy setups, this would be our first priority after stability is confirmed.

1 Like
19

I just got into experimenting with LED feedback with the ultimate goal to display a VU meter.
I can confirm that the stability is still a huge issue. I’m trying it with a BU16 on FW 1.2.36 and experience that the RX part stops working completely if MIDI CCs are received too quickly. There are also dropouts sometimes. This is my very basic code to test a mapping on the first 4 LEDs.

function scl(b, v)
	ret = math.floor(255 / b * v)
	return ret
end;
function led(a, b, c, d)
	led_value(0, 0, a)
	led_value(1, 0, b)
	led_value(2, 0, c)
	led_value(3, 0, d)
end;
if p2 < 32 then
	led(scl(32, p2), 0, 0, 0)
elseif p2 < 64 then
	led(255, scl(64, p2), 0, 0)
elseif p2 < 96 then
	led(255, 255, scl(96, p2), 0)
else
	led(255, 255, 255, scl(128, p2))
end;
print(cn)
cn = cn + 1

I’m sending CCs with a square wave LFO in Bitwig Studio to test the stability after I figured that sending CCs with a normal CC control breaks the RX functionality of the BU16 very quickly.

20

I think i’ll use the relative encoder approach for now