Bugs item #407793, was opened at 2001-03-12 02:12
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=110493&aid=407793&group_id=10493
Category: run-time
Group: None
Status: Closed
Resolution: Out of Date
Priority: 5
Submitted By: Brian D. Carlstrom (bdc)
Assigned to: Olin Shivers (olin-shivers)
Summary: external symbol lookups
Initial Comment:
From: "Robert E. Brown" <brown@grettir.bibliotech.com>
To: bdc@ai.mit.edu
Subject: external symbol lookups
Date: Tue, 5 Nov 96 00:52:43 -0500
OK, I took a peek at the Scheme code involved in
external symbol lookups.
The first thing I noticed is that lookup-all-externals
eventually calls
find-all-xs, which walks the pure and impure heap
areas looking for "external"
objects. This must be expensive! Once all the
external objects are found,
their addresses are computed and added to a table.
Luckily, the get-external lookup function is smart
enough to call
the external lookup function if the external it is
searching for is not
immediately found in the table. This means that the
table really does not
have to be filled with data when scsh resumes -- it
will be filled lazily
by calls to get-external.
Please try the following patch to startup.scm. It
turns off the immediate
filling of the external table.
demeter [73] diff -c startup.scm.~1~ startup.scm
*** startup.scm.~1~ Thu Oct 31 15:18:06 1996
--- startup.scm Tue Nov 5 00:38:31 1996
***************
*** 51,57 ****
(define (scsh-stand-alone-resumer start)
(usual-resumer ;sets up exceptions, interrupts,
and current input & output
(lambda (args) ; VM gives us our args, but
not our program.
! (init-scsh-hindbrain #t) ; Whatever. Relink &
install scsh's I/O system.
(call-with-current-continuation
(lambda (halt)
(set! %vm-prog-args (cons "scsh"
args)) ; WRONG -- use image.
--- 51,57 ----
(define (scsh-stand-alone-resumer start)
(usual-resumer ;sets up exceptions, interrupts,
and current input & output
(lambda (args) ; VM gives us our args, but
not our program.
! (init-scsh-hindbrain #f) ; Whatever. Relink &
install scsh's I/O system.
(call-with-current-continuation
(lambda (halt)
(set! %vm-prog-args (cons "scsh"
args)) ; WRONG -- use image.
With this patch, scsh on my machine now starts up in 5
seconds instead of 8.
A resumed "ekko" heap created with dump-scsh-program
now runs instantaneously.
It used to take 7 or 8 seconds to run.
I believe that most of the 5 seconds that scsh now
takes to start up are
related to reading the heap. I have not tried
static.scm with the above
patch. If I get a chance, I'll profile the heap
reading and other startup
actions to see if I can pare a little more time away.
By the way, Olin had posted a note to
comp.lang.scheme.scsh indicating that
he could start up scsh in 0.1 or 0.2 seconds on some
box. I am still curious
how he was able to achieve this!
bob
----------------------------------------------------------------------
>Comment By: Mike Sperber (sperber)
Date: 2002-09-03 14:50
Message:
Logged In: YES
user_id=43931
The code in question no longer exists in scsh 0.6.
----------------------------------------------------------------------
Comment By: Mike Sperber (sperber)
Date: 2002-09-03 14:49
Message:
Logged In: YES
user_id=43931
The code in question no longer exists in scsh 0.6.
----------------------------------------------------------------------
Comment By: Mike Sperber (sperber)
Date: 2002-09-03 14:48
Message:
Logged In: YES
user_id=43931
The code in question no longer exists in scsh 0.6.
----------------------------------------------------------------------
Comment By: Mike Sperber (sperber)
Date: 2002-09-03 12:16
Message:
Logged In: YES
user_id=43931
The code in question no longer exists in scsh 0.6.
----------------------------------------------------------------------
Comment By: Brian D. Carlstrom (bdc)
Date: 2001-03-14 04:20
Message:
Logged In: YES
user_id=27364
From: Olin Shivers <shivers@lambda.ai.mit.edu>
To: scsh-news@martigny.ai.mit.edu
Subject: Re: init-scsh-hindbrain
Date: 25 Feb 1998 00:04:45 -0500
brown@bibliotech.com (Robert E. Brown) writes:
> I've built scsh on a few different kinds of machines. At
least on NextStep
> and Solaris, the call to init-scsh-hindbrain in scsh-
stand-alone-resumer is
> responsible for about half of scsh's startup delay. If I
change the
> argument to init-scsh-hindbrain from #t to #f, then scsh
starts up much
> faster. Surprisingly, it also still seems to work.
>
> Can you explain what's going on and why the call is
necessary. Maybe it's
> not needed on all architectures or can be avoided under
some conditions.
> Thanks!
Yep, exactly right. What that boolean controls is the C
symbol relinking
step. That is, it causes Scheme 48 loop over its table of C
symbols
(which are used for the foreign-function interface, and all
the system
calls), and update its idea of the address at which each
symbol lives.
If the vm that resumes a heap image is the same one that
dumped it, you don't
have to relink the, because the old values are right. If
not, you will
die horribly on your first C call unless you relink. That
is why you are
getting away with it.
Here's the bitch of it: the way S48 does this relink step
is much, much slower
than it has to be. It does the system call (nlist) that
looks up a symbol once
for each symbol. But this system call takes a *vector* of
symbols to look up
-- you can do the entire relink in one system call, and
this would be much,
much, much faster.
I would love it if someone hacked scsh's code to support
this. I'm a little
backed up now; I don't have time myself.
Note that Kelsey may have fixed this in more recent Scheme
48 releases.
Scsh is *way* out of date on this spectrum. I had an
undergrad lined up
at MIT to port scsh over to Scheme 48 0.50, but he flaked
out, and I'm
currently without labor. It is something I would very much
like to do.
-Olin
----------------------------------------------------------------------
Comment By: Brian D. Carlstrom (bdc)
Date: 2001-03-13 04:24
Message:
Logged In: YES
user_id=27364
From: brown@bibliotech.com (Robert E. Brown)
To: scsh-news@martigny.ai.mit.edu
Subject: init-scsh-hindbrain
Date: 18 Feb 1998 22:10:18 -0500
I've built scsh on a few different kinds of machines. At
least on NextStep
and Solaris, the call to init-scsh-hindbrain in scsh-stand-
alone-resumer is
responsible for about half of scsh's startup delay. If I
change the
argument to init-scsh-hindbrain from #t to #f, then scsh
starts up much
faster. Surprisingly, it also still seems to work.
Can you explain what's going on and why the call is
necessary. Maybe it's not
needed on all architectures or can be avoided under some
conditions. Thanks!
bob
====================
(define (scsh-stand-alone-resumer start)
(usual-resumer ;sets up exceptions, interrupts, and
current input & output
(lambda (args) ; VM gives us our args, but not our
program.
===> (init-scsh-hindbrain #t) ; Whatever. Relink &
install scsh's I/O system.
(call-with-current-continuation
(lambda (halt)
(set! %vm-prog-args (cons "scsh" args)) ;
WRONG -- use image.
(set-command-line-args! %vm-prog-args)
(with-handler (simple-condition-handler halt
(error-output-port))
(lambda ()
(let ((exit-val (start (command-line))))
(if (integer? exit-val) exit-val
0))))))))) ; work around bug.
----------------------------------------------------------------------
Comment By: Olin Shivers (olin-shivers)
Date: 2001-03-12 02:36
Message:
Logged In: YES
user_id=101304
This is an old, known issue. Here is the problem with doing
lazy
initialisation: suppose I have a resolved external symbol in
my
running scsh -- it says that sin is at address 0x1234. I
dump out
an image, and then resume it with a *different* vm, where
sin is
as 0x4321. Oops. The "external" record for sin is already
resolved,
so a call to sin jumps off into hyperspace. Everyone dies.
Very sad.
Forcing all externals to be re-resolved at startup time
eliminates this
problem. But the find-all-xs technique for doing so is a
terrible one.
Rather, do one of these strategies:
- When writing out the image, write the externals in
their unresolved
form. At startup time, resolve them on demand.
- Keep all externals in a weak set. At startup time,
re-resolve them
all at once.
Clearly the former is the better strategy. It requires
hacking the image
dumper.
As a footnote, I'm *amazed* Robert Brown is getting 8 sec
startup
times. Oh, the humanity.
----------------------------------------------------------------------
Comment By: Brian D. Carlstrom (bdc)
Date: 2001-03-12 02:13
Message:
Logged In: YES
user_id=27364
From: "Robert E. Brown" <brown@grettir.bibliotech.com>
To: bdc@ai.mit.edu
Subject: startup times
Date: Tue, 5 Nov 96 02:10:31 -0500
To complete the story, I just generated a scsh.vm using the
official
scsh.image file. This static scsh starts instantaneously
the second
time it is run -- it takes a few seconds the first time.
The standard
/usr/local/lib/scsh/scshvm version always seems to take 3
seconds to
start, but that may be related to the fact that my heap
image is on
an NFS file server.
bob
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=110493&aid=407793&group_id=10493
|