As was mentioned in lecture yesterday, we want to try collecting and
collating/graphing your assignment 3 performance data as you generate
it.
This is for fun, not for any real purpose. (However, it does have a
use: it'll give you a sense of what "reasonable" performance numbers
for OS/161 are.) You will *not* be graded on how fast your system
runs, or on how much faster it gets while you work on it.
The results will be posted anonymously, so there's no reason to be shy
about participating. And don't wait until your system runs all the
tests before sending data; it's ok to say some of the tests don't
work. This is perfectly normal, after all, when the system's under
development.
The graphs will be posted here:
http://www.courses.fas.harvard.edu/~cs161/perf/
We will, as time permits, add in data from the solution set, previous
years' solution sets if they're different, and kernels submitted by
current and previous TFs. (Note that it is not the case that the TF
kernels are all fast or even bug-free, and the solution set
performance last year was strictly mediocre, so don't feel intimidated
by this.)
All the tests we'll be collecting data for are end-to-end VM
throughput benchmarks from src/testbin. The results in each case
should be the elapsed *simulator* time in seconds. (Don't use wall
clock time; it's different and depends on sys161's performance.)
To help measure only one thing, please run the tests with 'p' from the
kernel menu, not from your shell. This will print the proper elapsed
time. If you've changed this so that doesn't print timings any more,
please report equivalent timings - note that properly speaking this
should include the time taken by thread_fork and thread_exit.
Except as noted below, please use the default sys161.conf settings for
reporting these numbers, particularly the rpm setting of the swap
disk.
Ideally your numbers should be collected using optimized (-O2) kernels
(without "debug" in the kernel config). If your kernel doesn't work
that way or something, please do report unoptimized numbers instead,
but please tell us that they're from a debug kernel.
Ideally also your reported numbers should be the average of several
runs, with low standard deviations. While you should adhere to this
practice for your actual submitted performance writeup, please feel
free to send us less solid numbers for graphing purposes, particularly
early on. Just please tell us (briefly) what you did to get them.
These are the specific tests:
/testbin/matmult with 512k physical ram
/testbin/sort with 512k physical ram
/testbin/triplemat with 512k physical ram
/testbin/parallelvm with 1024k physical ram
Note that you should raise the RAM to run parallelvm. While it should
fit in 512k this year (last year, kernel stacks were 8k, and as
parallelvm forks 24 processes, this makes a big difference) it might
not, and in any event would take excessively long.
Here's a template for the information you should send us:
Group name:
Test: [matmult|sort|triplemat|parallelvm]
This test:
[ ] wasn't tried
[ ] crashes
[ ] deadlocks
[ ] fails or gets wrong answers
[ ] succeeds; time (in seconds) was:
The data came from:
[ ] an optimized (-O2) kernel
[ ] a debug (-g) kernel
The data is:
[ ] just a number
[ ] the average of five or more trials,
with standard deviation less than 1%
[ ] other:
If the traffic is high, or people prefer, I'll put a submission form
on the web page.
Note that you shouldn't do a lot of extra work to send us data; we're
assuming/hoping that these are either all numbers you'll be collecting
anyway, or numbers that you get for free when testing/debugging.
--
- David A. Holland / dholland(a)eecs.harvard.edu
Dear students:
There is a change with respect to this week's 7:30 PM section. Instead of
a regular section there is going to be a review session for the midterm.
The review has to be done early, because it needs to be taped for the
distance students, and we weren't given any convenient options for taping
it at an alternate time.
If you were going to attend the regular section at this time, don't
despair - this was going to be just a Q&A section. And if you have
questions,
feel free to make an appointment with me and ask your questions there.
Even though I realize that it's kind of early for you to be
thinking about the midterm at this point, I welcome everyone to attend
this midterm review, because it would be a pretty boring review without
anyone in the audience.
-- Sasha
Hi,
When read/write/execute are passed to as_define_region, are they all
supposed to be either 0 or 1? It seems weird to me that they would take
on any other values. I'm getting some weird behavior whenever I try to
run a program from the kernel command line. I try to run bin/true, and
I step through load_elf, and when I step into as_define_region, read is
passed in as 4, and write as 0. Is this normal behavior? I get the
feeling that it's not, because I later take a READONLY tlb fault on the
page that gets created there, so I assume write is supposed to be
something else. Anyone getting similar behavior?
Also, for the TLB functions, when we write an entry into the TLB, what
exactly is supposed to be in entryhi and entrylo? I know that the
virtual page number is supposed to be in entryhi, so is that just (vaddr
& PAGE_FRAME)? As for entrylo, I know it pertains to the physical page
number, but is it the page number relative to the bottom of useable
memory (i.e. just above the kernel heap and stack), or is it relative to
ALL physical memory?
Thanks a lot,
Brian
Because there's no debugger for userlevel code in OS/161 (unless you
write it, and we don't recommend you try) it will be to your advantage
to test your malloc implementation under Unix.
Because malloc is a standard C function and there's already a malloc
in every Unix's C library, this presents some problems. Furthermore,
the details vary depending on the flavor of Unix. So there's no
automatic way in the OS/161 tree to test your malloc this way; you'll
have to set this up yourself.
First, some notes:
(1) ICE is a 64-bit platform. If you want to test on ICE you'll
need to write portable code that will work properly in both a
32-bit and a 64-bit environment. Since you ought to be doing this
anyway, we expect you to cope. :-) Don't forget that alignment
requirements are greater on a 64-bit processor.
(2) If you use the OS/161 malloctest program for testing your
malloc on Unix, which you can, be very careful to set a limit on
the process heap size when running it. Otherwise, malloctest will
allocate huge amounts of memory and start the machine thrashing.
(Under Unix all the tests should run very quickly. If they don't,
probably you forgot to set the limit.)
To set the maximum process heap size with tcsh:
% limit memoryuse 1M
This limit is inherited by all processes run from that shell, so
the easiest thing to do is dedicate one window to running only
malloctest.
(3) If you use the OS/161 malloctest program for testing on Unix,
you will need to make a copy and hack the copy up a bit to get it
to compile. You may need to provide the "err..." functions; get
those from hostcompat. You may need to provide a type for
uintptr_t; if including <stdint.h> doesn't work, make a typedef to
define it to the unsigned integer that's the same size as a
pointer. On DU, that's unsigned long; elsewhere may vary.
(4) You can, or should be able to, test your malloc with any random
piece of Unix software you like, not just malloctest, if you
compile and link according to the following directions. This will
give your malloc code a much better workout than malloctest does.
There are two basic approaches to using your own malloc in Unix. One
is to link in both your malloc and the system malloc; the other is to
replace the system malloc and use only your own malloc. Neither one is
exactly easier to set up than the other; they're just different, and
they make different tradeoffs.
The following descriptions assume that your malloc and free code lives
in a file called "malloc.c".
Linking both mallocs
- ------------------
Because you can't have more than one symbol named "malloc" when you
actually link your program, you need to rename one. Since the system
one is out of your control, rename yours, using the C preprocessor.
Put the following at the beginning of your malloc.c and all source
files of your test program, *after* the standard #includes:
#include "mymalloc.h"
and then put the following in mymalloc.h:
void *mymalloc(size_t size);
void myfree(void *ptr);
#define malloc mymalloc
#define free myfree
This will rename your malloc and free routines so they don't conflict
with the system's malloc. Then you just compile the test program in
the ordinary way, including your malloc.c in it.
Advantages:
- straightforward.
- no weird linker options.
- no unexpected calls from odd places inside libc.
Disadvantages:
- might have to edit lots of files.
- the two mallocs in the system might start fighting each other.
Both your malloc and the system malloc will be calling sbrk() to get
memory from the OS. This is ok if nothing calls the system malloc, or
if everything that calls the system malloc finishes doing so before
yours gets used. However, if this condition isn't met, either or both
mallocs may become confused when pieces of the heap disappear from
under them, and strange things can (will) result.
Note that a lot of standard C library routines call malloc under the
covers, and you can't necessarily predict which in advance. So while
this technique will _probably_ work for malloctest, the more elaborate
the test program is, the more likely it is to fail.
Replacing the system malloc
- -------------------------
Here you need to get the system malloc to go away so you can provide
your own.
Because of various flaky properties of shared libraries, to make this
work reliably without various kinds of arcane fiddling, you need to do
a static link. This means you need to use the "-static" (or, on some
platforms, "-Bstatic") option when linking. BE SURE NOT TO FORGET
THIS, because if you forget, the link will still succeed but things
may not work... or may work only some of the time.
You don't need to do anything different when compiling, though. Just
add your malloc.c to the program in the normal way you'd do such a
thing.
Because of the way Unix linkers work, if there's some malloc-related
symbol the system expects that you aren't providing in your malloc.c,
the system malloc won't go away and you will get errors about malloc
and free being multiply defined. (Figuring out what symbol you're
missing at this point is involved and annoying and basically requires
using nm to search through the system libc's symbol table.)
On essentially all systems you will need to provide "realloc". On most
of them realloc won't actually be called unless your test code calls
it directly, so you can "implement" it with assert(0). Check the Unix
man page for the proper declaration for realloc.
On DU (that is, ICE) you will also need to provide a function called
__malloc_locks_reinit. It seems to be sufficient for it to be an empty
function that returns void and does nothing.
(I've tried BSD, but I haven't tried Linux recently, or OS X ever. If
you do try these, and they want more than "realloc", try providing
"valloc" as well. If "implementing" either of these as assert(0)
doesn't work, test on DU instead.)
Advantages of this technique:
- don't have to change anything in your test program(s).
- should always run properly once you get it to link correctly.
Disadvantages of this technique:
- less straightforward; requires more fiddling.
- requires extra linker options.
- you'll get called from odd places inside libc.
Because you're replacing the system malloc, your malloc will have to
field all memory allocation calls made by the C library. Some of these
may happen in places where making further C library calls isn't
particularly safe, such as when allocating the output buffer for
"stdout". Some of these may happen before main() is called. Etc. You
have to be careful what you do from inside malloc and free in this
environment to avoid accidental recursion.
Whether you adopt one or the other (or both) of these techniques, you
probably want to set up a special test area for them, and write
makefiles or scripts to do the compilation and linking of your malloc
tests. (But don't forget to arrange things so that changes to your
malloc.c get committed to the OS/161 tree in the proper place.)
Some additional malloc notes
- --------------------------
Because your malloc is part of the C standard library, it should not
pollute the linker namespace with irrelevant symbol names (either code
or data). (Hint: in C, symbols beginning with _ (underscore) are
reserved for the implementation. In OS/161, you're part of the
implementation.)
Don't forget that malloc must (by the C standard) always return
pointers that are sufficiently aligned that they can be used with any
data type.
--
- David A. Holland / dholland(a)eecs.harvard.edu
Hi-
I'm having trouble with two aspects of compiling/testing malloc on ice. I
think I followed Dave's directions, but I'm still having issues.
First of all, the Makefile for malloctest doesn't create a host-malloctest
version to run on digital unix. And second, my code for malloc in
src/lib/libc/malloc.c doesn't seem to be reading the stdlib.h file in
src/include (or it is reading an old version of it). Anyway, any
suggestions on getting this to work?
Nick
--
Nicholas Shiftan
Division of Engineering and Applied Sciences
Harvard University
shiftan(a)fas.harvard.edu
(619) 540-5606
365 Dunster Mail Center
Cambridge, MA 02138
hello cs161ers:
it has come to light that some of you included a "feature" in your shell which
appends "/bin/" to the command entered on the command line.
so
myshell> cp
is executed as /bin/cp
this may have seemed like a good idea at the time, but a) it doesn't allow you
to run things from /testbin/ (where a lot of our tests live) and b) it may
break some of our testing scripts.
if you did this, please email cs161@fas and let us know.
-gwa-
If anyone (who has submitted assignment 2) wants a copy of the
solution set to get started on assignment 3, please mail cs161@fas.
--
- David A. Holland / dholland(a)eecs.harvard.edu
If you intend to watch the section video of 03/04, please use the attached
section notes: the section notes weren't posted together with the video.
-- Sasha
