Simultaneous address update and read for external SRAM

[atoone]

I'm interfacing an external SRAM to a Lattice part, and coming up against a timing issue that makes the design unstable. Frustratingly, the simple cases work fine, with problems emerging as the overall design gets larger.

Essentially, I'm doing a continuous lookup, where the results of the previous read are used to set the address of the next read. Something a bit like this (other cases and additional computation removed).

always @(posedge CLOCK) begin
    case( state )
       PHASE_ONE: begin
            ram_address <= location;                   // First phase - lookup stable address
            some_value <= ram_data[15:8];          // Get result of previous PHASE_TWO <- This is unstable
            location++;
            state <= PHASE_TWO;
          end
       PHASE_TWO: begin
            ram_address <= { 7'b0, ram_data[7:0] };  // Second phase - lookup based on previous value
            other_value <= ram_data[15:8];               // Also use previous value for a calculation <- This is stable
            state <= PHASE_ONE;
          end
    ...
end  

Output enable, chip select and so on are all held continuously active (low).

This works just fine... some of the time (and of course, always in simulation). Then small changes elsewhere in the project knock everything out of whack. The overall clock rate is well within the limits of the SRAM.

My assumption (hard to verify) is that setting the ram_address with it's own data is the root of the problem, since the other_value read in phase_two appears stable, and both phases do the same simultaneous set address and read data functions. When the project is reduced to bare bones, this all works, but it becomes unstable as other functionality is added.

Any suggestions for checking/debugging/diagnosing the issue? I really want to avoid breaking this out into multiple steps as it dramatically reduces the available bandwidth..

The ram interface is declared as a module that takes the ram_address and control signals and mediates the bidirectional bus to the physical device (somewhat simplified):

    input   [ADDRESS_WIDTH-1:0] address;  // Address in
    output  [DATA_WIDTH-1:0]    data_read; // Data from SRAM
    input    [DATA_WIDTH-1:0]   write_data;  // Data to SRAM
    
    input   write_en;
    ...
    output  [ADDRESS_WIDTH-1:0] sram_addr;   // External SRAM interface
    inout   [DATA_WIDTH-1:0] sram_data;
 
    ...
    
    assign data_read = sram_data;

    assign sram_data = write_en ? write_data : 16'bz;

    assign sram_addr = address;

In general, how are people getting help for larger projects, when it moves beyond the 'casual hacking' phase? Since this runs on custom hardware, it's not really something that can be usefully explored by other developers on a whim, so it might be helpful to find advisors who can perhaps handle a slightly more formal arrangement..

[atoone]

Further investigation has isolated the problem to this line:

ram_address <= { 7'b0, ram_data[7:0] };

I can store the value to another register and simultaneously update the ram_address on the same clock, but setting the ram_address from the incoming data becomes unstable once the rest of the design is more complex.

[atoone]

I'm continuing to explore this issue. The static RAM chip is more than capable of continuous reads

Even without using the value in one read to control the address in the next, if the address selection logic is moderately complex, then reading a value and updating the address on the same clock cycle can cause metastability depending on how Yosys synthesises the logic.

assign ram_address = <some form of arbitration logic>

always @(posedge clk) begin
   desired_address <= next value;
   some_register <= ram_data_output;

   next_value <= next_value+1;
end

This appears to be independent of clock speed. Depending on the tool's mood, the resulting binary can be either rock solid or consistently unstable. The problem is all of the solutions I can think of involve halving the RAM bandwidth (e.g. set up address on one clock, read value on the next) - with 50% of the time just spent idling.

Is there a reliable way to ensure the sequence (and avoid any unwanted optimisation) of read-value -> update-address that doesn't involve clock domain crossing, or unnecessarily long idle cycles?

A rough illustration of the timing (times in nanoseconds):

[rowanG077]

Are you using IO registers for this? If not the placement and routing will throw everything off.

[atoone]

Ah. I'm not - instead just assigning wires to the package pins. Is there any information on how the timings interact?

Does using an IO register force the placement/routing to some particular time constraints?

If it's an unclocked IO register, is there any difference from using a straight pin/wire assign? And if it's clocked does that mean the I/O will be 1 cycle behind? What about cross domain timings?

(Sorry, lots of questions)

[rowanG077]

Does using an IO register force the placement/routing to some particular time constraints?

Yes it uses a special register that is close to the actual IO pad of the FPGA, which means consistent timing. And from what you are describing this is exactly what you are struggling with.

If it's an unclocked IO register, is there any difference from using a straight pin/wire assign? And if it's clocked does that mean the I/O will be 1 cycle behind? What about cross domain timings?

Registers are clocked by definition, so yeah just like normal register they introduce a clock cycle latency. They should be clocked on the same domain as the source.

Depending on the actual timing figures you should probably do:

• Use IO registers, both input and output. Compensate in your actual logic for this additional clock cycle delay.
• If you can't hit the setuphold window of your SRAM AND receiver IO registers on the FPGA you have to adjust the timing. This can be done in two ways. Either create an additional clock that is routed to the SRAM that is phase shifted from your main clock, you can phase shift it enough so both setup hold windows are respected. Or you can use FPGA build in delay cells that can delay a signal going out of the FPGA or coming into the FPGA. Depending on your needs one or the other can be more convenient.

[atoone]

I can't thank you enough for your help - it's really appreciated.

Registers are clocked by definition

Good point - I had thought of the lattice SB_IO #( primitives as registers, but they can be unclocked (and therefore, by definition, not registers!) - at which point they confer no useful benefit over plain assignment if I understand correctly.

From what you've said, using clocked address and data IO registers would introduce a two clock delay from address select -> data, so I'd need to do some fairly smart interleaving of reads so that later reads can use data previously requested... ugly, but I can see how it'd work.

I'm pretty certain I'm well within the general setup and hold times of both RAM and FPGA: crude read and write loops work at significantly higher clock frequencies. The problem comes when the additional logic is added to process the data in a slightly smarter way, so I just need to get the pipeline right to keep things happy.

Please forgive me if I come back in a couple of days time to ask more stupid questions!

[whitequark]

The problem comes when the additional logic is added to process the data in a slightly smarter way,

The rewriting of logic under equivalence rules that do not guarantee the absence of glitches or consistent timing is both the great benefit and (in a case like this) a weakness of synthesis tools. It is inherent to their design; if you want asynchronous logic to have consistent timing you will, in general, need to instantiate the LUTs yourself, and then quite possibly constrain them to specific BELs. Most of modern digital design happens in synchronous domains (the globally-asynchronous, locally-synchronous model) and this is what the tools are tailored towards.