Testing of the Cache Memory in the

OpenSPARC T1

Matteo Ainardi - 147695

Matteo Bosio - 148451
Salvatore Campione - 145781
Vittorio Giovara - 149374
Alberto Grand - 149389

June 20, 2008

Contents

1 Introduction 2

2 Synthesize the OpenSPARC processor T1 3

2.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.2 First approach: Basic processor in Modelsim . . . . . . . . . . . 4
2.3 Second approach: Full processor in Synopsys . . . . . . . . . . . 4

3 The Sparc Cross Compiler 6

3.1 What is the Cross Compilation . . . . . . . . . . . . . . . . . . . 6
3.2 The gcc-sparc-linux compiler . . . . . . . . . . . . . . . . . . . 6
3.2.1 Build explanation . . . . . . . . . . . . . . . . . . . . . . 6
3.2.2 The building process . . . . . . . . . . . . . . . . . . . . . 7
3.3 Environment Loading . . . . . . . . . . . . . . . . . . . . . . . . 8

4 Building blocks of the OpenSPARC T1 core 10

Bibliography 12

1
Chapter 1

Introduction

Our team was assigned to generate a working testbench for evaluation and
testing purpose of the memory cache of the OpenSPARC T1.
The OpenSPARC T1 is a completely open source processor, produced and
supported by SUN. Because of the availability of the source code, our plan was
to synthesize the core processor and perform a simulation of the cache using
special instruction sequence obtained by the use of a Sparc cross compiler; in
order to understand the correct sequence of instructions a building block scheme
of the processor was to be realized.
Unfortunately we encountered many difficulties during the development of
the plan as the task proved excessively difficult, especially the synthesis part,
which has not been fully carried out.
In this document we outline the final status of the project, the step we have
followed and the results of the operations.

2
Chapter 2

Synthesize the OpenSPARC

processor T1

2.1 Overview
The OpenSPARC material (which comprises documentation, verilog sources and
testbenches) can be downloaded freely from the web at [1].
The processor can be synthesized in two different ways: core1 configuration
and chip8 configuation.
The core1 environment consists of:

1. One SPARC CPU core

2. Cache

3. Memory

4. Crossbar

The chip8 environment consists of:

1. A full OpenSPARC T1 chip, including all eight cores

2. Cache

3. Memory

4. Crossbar

5. I/O subsystem

Each of the previous configurations can be obtained by synthesizing different

verilog files of the project.

3
2.2 First approach: Basic processor in Model-
sim
The first expected result consists of obtaining a complete functional version of
the processor in its basic functions; for this reason, we tried to compile and
synthesize a minimum configuration set of the processor using ModelSIM. After
creating a new project in ModelSIM and inserting all the files (both the Verilog
files and the needed libraries) inside it, we started with a syntax check on the
code, preparing for the simulation.
Unfortunately, a lot of errors appeared. We recognized some errors were due
to missing includes and so we tried to fix them; however, after overcoming the
syntax errors, we faced major problems in the simulation part.
When trying to simulate we had to cope with technological library problems
(i.e. no hardware basic blocks matched the requested) since the code was written
for synopsys, as we found out after thorough inspection.
Since we noticed that we wouldn’t have reached a complete simulation, we
decided to change the configuration environment.

2.3 Second approach: Full processor in Synop-

sys
First of all, we logged on the Politecnico server used for Synopsys synthesis
at mordor.polito.it To start the synthesis process, we first had to set the
environment variables for the machine we were using. Script.sh is shown
below:

#!/usr/bin/bash

export DV_ROOT="/cadtools/opensparc"
export MODEL_DIR="/cadtools/opensparc/model"
export CC_BIN="/usr/dist/pkgs/sunstudio_‘uname -p‘/SUNWspro/bin"
export SYN_HOME="/cadtools/synopsys/2002.05-SP1";
export LM_LICENSE_FILE="/import/EDAtools/licenses/synopsys_key:
/import/EDAtools/licenses/ncverilog_key";
export TRE_ENTRY="/";
export TRE_LOG="nobody";
export TRE_SEARCH="$DV_ROOT/tools/env/tools.iver";
export ENVDIR="$DV_ROOT/tools/env";
export PERL_MODULE_BASE="$DV_ROOT/tools/perlmod";
export SYN_LIB="$SYN_HOME/libraries/syn";
export SYN_BIN="$SYN_HOME/sparcOS5/syn/bin";
export PERL_VER="5.8.7";
export PERL_PATH="$DV_ROOT/tools/perl-$PERL_VER";
export PERL5_PATH="$DV_ROOT/tools/perl-$PERL_VER/lib/perl5";
export PERL_CMD="$PERL_PATH/bin/perl";
export PATH="$DV_ROOT/tools/bin:$NCV_HOME/tools/bin:$VCS_HOME/bin
:$VERA_HOME/bin:$SYN_BIN/:$CC_BIN/:$PATH";

4
 The synthesis process was started by the command rsyn -all; we had to
disable the server connection timeout, since it took several hours to accomplish
the task (about 23 hours).
In the end, no error messages appeared, however, when we verified the
synthesis output, only some blocks were successfully synthesized, while others
showed some errors.
The output of the synthesis is located in the following path (of the server)
/cadtools/opensparc/design/sys/iop; it is possible to recognise the syn-
thetized modules by looking in the synoptis/gate directory which contains
the physical descritption of the path gate.
At this point we decided (together with the teacher assistant) to give up
since we had no idea how to continue the workflow.

5
Chapter 3

The Sparc Cross Compiler

3.1 What is the Cross Compilation

Perfoming Cross Compilation is the technique for creating executable code for
a platform different from the platform the code is generated on; the software
that performs cross compilation is called Cross Compiler.
Hereinafter the machine on which the compiler is located will be called host,
the machine on which the code will run target.
This special compiler must be compiled with specific instruction on the target
architecture; moreover, the standard libraries must already be available both for
the target and host machine. When building a cross compiler it is intended that
the whole suite of programs are built, including the assembler, the binutils1 the
core compiler, the language extensions (optional) and the kernel environment
(of the host platform).

3.2 The gcc-sparc-linux compiler

As the GNU GCC is one of the most diffused compilers, with extremely good
documentation, not counting being open source, it has been chosen as starting
point to obtain the SPARC cross compiler. The host machine run Gentoo
Linux since it is one of the Linux distrubutions most suited for compilation and
development.
Compiling the GCC is rather difficult, and its cross versions are none easier:
however with some command line tuning it is possibile to sequencially build all
the needed parts.

3.2.1 Build explanation

As a clean build of the compiler is require, the sysroot approach is used: the
GCC will consider the source directory ad the root of a tree that contains a
(subset of) the root filesystem of the target operating system, requiring that
target system headers, libraries and run-time objects are placed there.
It is important to note that both the GCC and the libraries must be compiled
twice, particially and completely, in two different occasions. This is because of
1 binutils are a set of tools for creating, analyising and using custom libraries

6
dependecies among the GCC toolchain and the libraries that need to be satisfied
before attempting any build.
Having said that, in order to obtain a working cross compiler, it is necessary
to build the following software in this order:

1. Binutils

2. Kernel headers

3. Glibc headers

4. GCC stage1

5. Glibc

6. GCC stage2

Gcc stage1 includes only the C language while stage2 comprises all the other
languages (C/C++/Java/ADA etc). For a correct configuration the compiler
used for building the cross compiler must support the -mlong-double-128 flag
(obtained by configuring GCC with --with-long-double-128).

3.2.2 The building process

Before beginning, it is important to setup two environment variables, that will
faciliate code reproduction in the following steps; the first one set the target
architecture for the programas, the second one is for the kernel architecture.

$ export CTARGET=sparc-linux
$ export ARCH=sparc

First of all, binutils must be configured, built and installed, taking care of
removing unnecessary libraries.

$ cd /path_to_binutils_source/
$ ./configure --target=$CTARGET --prefix=/usr --with-sysroot=/usr/$CTARGET
$ make
$ make install DESTDIR=$PWD/install-root
$ rm -rf install-root/usr/{info,lib,man,share}
# cp -a install-root/* /

Then compile the kernel, even though it will result in an error, as the gcc
cross compiler isn’t ready yet, but what is really needed are the kernel headers.

$ cd /usr/src/linux
$ yes "" | make ARCH=$ARCH oldconfig prepare
# mkdir -p /usr/$CTARGET/usr/include
# cp -a include/linux include/asm-generic /usr/$CTARGET/usr/include
# cp -a include/asm-$ARCH /usr/$CTARGET/usr/include/asm

The GNU C Libraries (Glibc) suffers from the same problem as above, but
yet again only the headers are needed at this time.

7
$ cd /path_to:glibc_source/
$ mkdir build
$ cd build
$ ../configure --host=$CTARGET --prefix=/usr --with-headers= \
/usr/$CTARGET/usr/include --disable-sanity-checks
# make -k install-headers install_root=/usr/$CTARGET
# mkdir -p /usr/$CTARGET/usr/include/gnu
# touch /usr/$CTARGET/usr/include/gnu/stubs.h
# cp bits/stdio_lim.h /usr/$CTARGET/usr/include/bits/

Now it is possibile to build the core of the GCC compiler (stage1), linking
the libraries for the target architecture and removing the unnecessary ones.

cd /path_to_gcc_source/
# ln -s usr/include /usr/sparc/sys-include
$ mkdir build
$ cd build
$ ../configure --target=sparc --prefix=/usr --with-sysroot=/usr/sparc \
--enable-languages=c --disable-shared --disable-checking \
--disable-werror --disable-libmudflap --disable-libssp
$ make
$ make install DESTDIR=$PWD/install-root
$ rm -rf install-root/usr/{info,include,lib/libiberty.a,man,share}
# cp -a install-root/* /

After this step, the C libraries can be completely compile,

$ rm -rf build
$ mkdir build
$ cd build
$ ../configure --host=$CTARGET --prefix=/usr
$ make
# make install install_root=/usr/$CTARGET

as well as the GCC, including any other language (stage2).

$ ./configure --target=$CTARGET --prefix=/usr --with-sysroot=/usr/$CTARGET \

--enable-languages=c,c++ --enable-shared --disable-checking \
--disable-werror
$ make
# make install

At this point, if no error occurred, the cross compiler can be invoked from
command line with gcc-sparc-linux.

3.3 Environment Loading

One final section of code should be produced, separately from the compiler:
the loader, which takes care of storing the generated code into the instruction
memory which subsequentially loads it in the processor registers, the Arithmetic
Logic Unit and the memory cache.

8
 The loader can be implemented with any hardware description language
like VHDL or Verilog, using a read from file function and converting data into
bus values. However this procedure requires high level of details of the final
architecture, as well as the synthezised unit under test, for some important
tuning; since neither of these were available it has been not possible to write
any memory loader.

9
Chapter 4

Building blocks of the

OpenSPARC T1 core

For the purposes of designing a test-bench for the cache memory of the OpenSPARC
T1 processor, an in-depth knowledge of its architecture and of the building
blocks it comprises is required. One of the teams in our group was in charge of
producing a block scheme of the OpenSPARC core having such a level of detail
as to permit a test design.
Our first reference was the OpenSPARC T1 Microarchitecture Specifica-
tion, available on the OpenSPARC website. It provides a good overview of the
architecture, as well as more detailed information about its implementation.
Nonetheless, the description was at times a rather high-level and qualitative
one and only scratched the surface of some components, which was usually not
enough for our goal.
As a consequence, we later decided to take a closer look at the Verilog source
code available at [4]. By means of the sidebar it is possible to navigate through
the source code. Since we were only interested in the architecture of a single
core, we started our inspection from the cmp top → OpenSPARC T1 → sparc
module within the source browser hierarchy and we progressively analysed all
the lower-level source files.
The following criterion was adopted to choose only the most meaningful
signals among the many:

• data buses were added to the block scheme if their width was relevant
(usually larger than 6 or 7 bits) or if their presence was required for some
other reason;

• control signals were usually not inserted in the block scheme;

• scan-in/-out signals for test purposes were included.

The block scheme we obtained is depicted in the following figure.

10
Figure 4.1: Building blocks of the OpenSPARC T1 core

11
Bibliography

[1] OpenSPARC Official Site

http://www.opensparc.net

[2] Wikipedia, the free encyclopedia, Cross compiler,

http://en.wikipedia.org/wiki/Cross_compiler

[3] Mike “Vapier” Frysinger, Cross Compils Guts,

http://dev.gentoo.org/~vapier/CROSS-COMPILE-GUTS

[4] OpenSPARC Verilog Hierarchy

http://opensparc-t1.sunsource.net/nonav/source/verilog/html/
verilog.html

12