CS355 Sylabus

Solving the Read after Write Data Hazard in the LOAD instruction

Data Forwarding Hardware for LOAD instructions
- Let's try to pull the same "data forwarding" trick on the LOAD instruction and see what kind of millage we are getting...
  Through additional data forwarding circuitry, we aim to make the value in LMDR avaliable to the EX stage (where the value is need to execute the ADD (or another ALU) instruction.
  Because the value in the LMDR will be used to update the register in the next CPU cycle, we do not need to retain the value using additional registers - so there is no need to add forwarding registers.
  We only need to feed the LMDR value back to the EX stage and add "intelligent selection" hardware to determine when it is appropriate to use the value from the LMDR register.
- Let us first examine the hardware to forward the data from the LMDR to the EX stage:
The data forwarding circuitry to solve the LOAD instructon conists of:
1. Wiring to direct the output of LMDR register to the input multiplexors in the EX stage.
2. Additional selection hardware in the input multiplexor that determine when the value on LMDR must be used.

Additional selection logic needed

The selection logic of for the first source operand (the yellow mux in the above figure) implements the follow selection function:

if ( Instruction == Branch ) select PC1 as operand else if ( Dest Reg in IR(WB) == Src1 ) select LMDR as operand else if ( tag1 == Src1 ) select ForwReg1 as operand else if ( tag2 == Src1 ) select ForwReg2 as operand else select A as operand

To understand why we use "Dest Reg in IR(WB)" in the test, it is important to pay attention to the location of the LOAD instruction when the fetched value is in the LMDR register.

Take a quick look at the following picture that was from the previous webpage that depicts the situation when the value 4000 was fetched from memory and about to be written to register R1:

You can clearly see that the LOAD instruction is contained in the instruction register IR(WB), so that's why we need to use ""Dest Reg in IR(WB)" in the selection test.

The second multiplexor (gold colored one) will select from among: IR1 (a constant), B, ForwReg1 and ForwReg2.

The selection logic of this multiplexor is as follows:

if ( Instruction Imm bit is set to 1 ) select IR1 as operand else if ( Dest Reg in IR(WB) == Src2 ) select LMDR as operand else if ( tag1 == Src2 ) select ForwReg1 as operand else if ( tag2 == Src2 ) select ForwReg2 as operand else select B as operand

The reason for this test is similar to the first case.

But if you look carefully at the above figure, you will discover that the read after write data hazard caused by the LOAD instruction is more severe than the one caused by an ALU instruction:

If we permit the instruction to proceed unstalled, then: at the moment that the LOAD instruction has fetched the value 4000 into LMDR, the "ADD R4,R1,R4" instruction that follows the LOAD instruction has completed the computation R4+R1
So there is no way that the "ADD R4,R1,R4" instruction is able to make use of the forwarded value even after we have added the circuitry.
So are we sunk ???

Solution

We are not sunk. It only means that we cannot run the pipeline as fast as possible
To solve this obnoxious data dependency problem, we have to sacrifice some speed by not allowing the "ADD R4,R1,R4" instruction to proceed (from the EX stage) into the MEM stage when the LOAD instruction is fetching the future value of R1 from memory.
When must we stop the instruction in the EX stage from executing:
- Instruction in the MEM stage is a LOAD instruction
- Instruction in the EX stage uses the the destination register as a source.
The combinatorial logic of this decision can be formulated as follows:

if ( IR(MEM)'s LD bit == ONE // Load instr in MEM and IR(EX)'s BRANCH bit == ZERO // ALU, LD or ST instr in EX and ( IR(EX).Src1 == IR(MEM).Dest or IR(EX).Src2 == IR(MEM).Dest ) ) then select STALL EX (and ID) stages else select do not stall EX (and ID) stages

Notice from the previous discussion (see: click here ), the IF stage must be stalled when a LOAD instruction is in the MEM stage (because the MEM stage will be using the system bus). So we don't need to add additional circuitry to stall IF stage, only for the the EX and ID stages.
The following figure contains the necessary circuitry to detect the following detect the situation that we must stall the EX stage:

The added circuit simply compute the logical function given above:
STALL = (IR(MEM).LoadBit == 1) AND (IR(EX).BranchBit == 0) AND ( IR(EX).Src1 == IR(MEM).Dest OR IR(EX).Src2 == IR(MEM).Dest )

Let us now see how the improved pipelined CPU correctly handle the read after write data hazard caused by the LOAD instruction

Runing the LOAD example through the improved pipelined CPU
- Reconsider the following program that is executed by the IMPROVED pipeline:
```
   LD  [R2+R3], R1            R2=11, R3=9, R4=1, R5=8, R6=0, R7=2
   ADD R4, R1, R4
   ADD R5, R1, R5
   ADD R6, R1, R6
   ADD R7, R1, R7
   ...
```
- Recall that the correct behavior (one that an assembler programmer would expect) is: R1 := Memory[R2+R3] = 4000 (assume that memory location 20 contains the value 4000), then the other instructions will add R1=4000 to registers R4 (4000+1), R5 (4000+8), R6 (4000+0) and R7
CPU Cycle 1
- At start of the CPU cycle, the IF stage sends out PC
- At end of the CPU cycle, the IR(ID) register is updated with the instruction fetched (LD [R2+R3], R1)
- The picture above depicts the content of the CPU at end of the first CPU cycle (and the start of the 2nd cycle) - same as before, the interesting stuff happens much later....
CPU Cycle 2
- At start of the CPU cycle, the ID stage sends out selection signal that selects values from R2 and R3
- At end of the CPU cycle, A register is updated with R2 = 11, B register is updated with R3= 9.
- Also, at the end of the CPU cycle, the instruction (LD [R2+R3], R1) is moved into IR(EX) and instruction ADD R4, R1, R4 is fetched into IR(ID)
- The picture above depicts the content of the CPU at end of the second CPU cycle (and the start of the 3rd cycle) - still the same as before as the interesting thing will happen later....
CPU Cycle 3
- At start of the CPU cycle, the EX stage selects values from R2 and R3 for the ALU, use the ALU opcode to make ALU add the input values forming the result 20
  Also, at start of the CPU cycle, the ID stage selects R4 (1) and R1 (123) to be copied into the A and B registers.
- At end of the CPU cycle, ALUo and DMAR registers is updated with the value R1+R2 = 20 (future value of R1)
  Also, at the end of the CPU cycle, the instruction (LD [R2+R3], R1) is moved into IR(MEM), ADD R4, R1, R4 is moved into IR(EX) and instruction ADD R5, R1, R5 is fetched into IR(ID)
- The picture above depicts the content of the CPU at end of the 3rd CPU cycle (and the start of the 4th cycle) - still the same as before, but the interesting things will happen right now.... because the LOAD instruction in the MEM stage.
CPU Cycle 4
- At start of the CPU cycle, the address in LMAR (= 20) is sent on the address bus and the value in the memory location 20 is being fetched. This value will arrive towards the end of the CPU cycle (because the memory is pretty slow).
- The STALL detection hardware detects that the instruction in MEM stage is a LOAD instruction while some non-branching instruction in the EX stage is trying to use the result of the LOAD instruction right away.
  The STALL detection hardware will issue a STALL signal to the EX and ID stages
  NOTE: the EX stage will perform the normal computation (you can't stop the ALU from doing that and don't need to). At the end, the result of the ALU will simply be ignored and not be used to update the ALUo and LMAR. The most important effect of the stall is the instruction in the EX stage is NOT changed
  NOTE: the ID stage will also perform its normal computation - the result fetched from the registers will not be used to update the A and B registers. The most important effect of the stall is the instruction in the ID stage is NOT changed
- At end of the CPU cycle, LMDR register in MEM stage is updated with the value fetched from memory (4000).
  Also, at the end of the CPU cycle, the instruction (LD [R2+R3], R1) is moved into IR(WB), ADD R4, R1, R4 is RETAINED in IR(EX), instruction ADD R5, R1, R5 is RETAINED in IR(ID)
  - The picture above depicts the content of the CPU at end of the 4th CPU cycle (and the start of the 5th cycle)
  - Notice that a NOP (harmless) instruction must be inserted into IR(MEM) to ensure that no valueable information is inadvertently altered by the CPU stages. A similar technique as that one used in the IF stage is used (see click here )
CPU Cycle 5
- At start of the CPU cycle, the LOAD instruction is in the WB stage and because the destination register is equal to the Src2 register in the EX stage, the new (additional) forwarding hardware ( click here ) will detect the dependent and uses the value in LMDR as second operand.
  So the correct value of R1 will be used by the first ADD instruction
- At the start of the CPU cycle, the old value of register R1 is being fetched but this old value will be REPLACED by the new value 4000 of register R1 after the middle of the CPU cycle !!!
  That is because the way the general purpose registers are upodated. Recall the timing of the register updates ( click here ):
  - General purpose registers are updated in the middle of the CPU cycle
  - The other registers (including the A and B registers in the ID stage) are updated at the end of the CPU cycle.
  So the correct value will be fetched (and later used) by the second ADD instruction.
  The following picture illustrates what is happening just before the end of the 5th CPU cycle:
- At the end of the cycle, ALUo will be updated with 4000+8 = 4008, which is the CORRECT outcome because R1 will be equal to 4000).
- Also at the end of the cycle, the B register will contain 4000 which is the CORRECT outcome because R1 is equal to 4000).
- At the end of the cycle, the result of the ALU (4008) is copied into Forwarding Register 1 along with the register tag "100" indicating register R4 - this Forwarding Register value will aid in "correcting" a subsequent instruction that use R4 as a source operand.
- The picture above depicts the content of the CPU at end of the 5th CPU cycle (and the start of the 6th cycle)
- Clearly, the ADD instructions following the LOAD instruction are processed correctly (i.e., they will be using the correct value for R1)
Smart Multiplexors used in Data Forwarding
- We need to add one more selection stage in the input multiplexors in the EX stage.
- The following circuit diagram shows the multiplexor for the first input of the ALU (the yellow multiplexor):
The MUX for the second source operand of the ALU is constructed in a similar manner.
There are still other types data hazards - it is not interesting to go through all possible data hazards.
I have used the two examples to teach you the principles on how to solve data hazard problems:
1. Find the cause (you would need to understand the operation of the pipeline CPU very well to do this).
2. Determine the severeness (limit the cost of the solution)
3. Add forwarding circuitry to redirect the needed value as quickly as possible.
4. Add memory to retain the value if needed (forwarding registers) - additional memory may not be necessary.
5. Stall the pipeline if you cannot return the needed data in time.