From rodney.bates at wichita.edu  Tue Nov  4 15:45:35 2008
From: rodney.bates at wichita.edu (Rodney M. Bates)
Date: Tue, 04 Nov 2008 08:45:35 -0600
Subject: [M3devel] New m3gdb documentation
Message-ID: <4910600F.7000009@wichita.edu>

I have new and greatly expanded documentation on m3gdb.  It is created
in docbook format, and I have managed to get it to convert to one-page
html.  I intend to do multi-page html and pdf also.

I am not sure where in the repository this should go, where to put html
files so they are available on the web, or what to do with mechanically
derived files in the repository.  I thought of creating a new subdirectory
doc/help/m3gdb, but I can only find how to get cvs to add a new ordinary
file, not a directory.

Also, I presume that Olaf will want to get the html to look like the other
pages, which some kind of style sheet should do, but I don't know where to
find that either.

Any help will be appreciated.

Rodney Bates


From wagner at elegosoft.com  Tue Nov  4 17:04:32 2008
From: wagner at elegosoft.com (Olaf Wagner)
Date: Tue, 04 Nov 2008 17:04:32 +0100
Subject: [M3devel] New m3gdb documentation
In-Reply-To: <4910600F.7000009@wichita.edu>
References: <4910600F.7000009@wichita.edu>
Message-ID: <20081104170432.1nn4yh39444sc48k@mail.elegosoft.com>

Quoting "Rodney M. Bates" <rodney.bates at wichita.edu>:

> I have new and greatly expanded documentation on m3gdb.  It is created
> in docbook format, and I have managed to get it to convert to one-page
> html.  I intend to do multi-page html and pdf also.

All documentation is checked-in below cm3/doc and cm3/www.
m3gdb should probably be integrated into doc.

> I am not sure where in the repository this should go, where to put html
> files so they are available on the web, or what to do with mechanically
> derived files in the repository.  I thought of creating a new subdirectory
> doc/help/m3gdb, but I can only find how to get cvs to add a new ordinary
> file, not a directory.

just use `cvs add <dir>'

> Also, I presume that Olaf will want to get the html to look like the other
> pages, which some kind of style sheet should do, but I don't know where to
> find that either.

The style sheet is normal.css at the top level.
You can reference it like this in the HTML header:

     <LINK HREF="normal.css" REL="stylesheet" TYPE="text/css">

It's checked in at cm3/www/normal.css.

When you've checked-in all your additions, we need to ship them
to the WWW server to make them visible. I can do that if you like,
just let me know when you're finished.

It's great that you're taking care of m3gdb and thus providing a
working debugger for all M3 users!

Regards,

Olaf

> Any help will be appreciated.
>
> Rodney Bates
-- 
Olaf Wagner -- elego Software Solutions GmbH
                Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany
phone: +49 30 23 45 86 96  mobile: +49 177 2345 869  fax: +49 30 23 45 86 95
    http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin
Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194



From rodney.bates at wichita.edu  Thu Nov  6 17:31:23 2008
From: rodney.bates at wichita.edu (Rodney M. Bates)
Date: Thu, 06 Nov 2008 10:31:23 -0600
Subject: [M3devel] New m3gdb documentation
Message-ID: <49131BDB.20903@wichita.edu>

There is a new and greatly expanded web page with documentation on m3gdb,
the Modula-3 debugger, located at:

   http://modula3.elegosoft.com/cm3/doc/help/m3gdb/m3gdb-onepage.html

It's also accessible with more indexing, through the normal Modula-3
web pages, for example, starting at:

   http://modula3.elegosoft.com/cm3

and clicking through:

   -> Documentation -> CM3 Operations Guide-> m3gdb

Rodney M. Bates




From jay.krell at cornell.edu  Sat Nov  8 16:40:50 2008
From: jay.krell at cornell.edu (Jay)
Date: Sat, 8 Nov 2008 15:40:50 +0000
Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
Message-ID: <COL101-W11F245FB5F2B72E83291B8E6180@phx.gbl>


Tony, how about switching these #if's?
I explain some of why below.

parse.c:

static void
m3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T)
{
  tree val;
#if 0
  if (o != 0 || TREE_TYPE (v) != dst_t) {
    v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),
     bitsize_int (o));
  }
#else
  /* failsafe, but inefficient */
  if (o != 0 || TREE_TYPE (v) != dst_t) {
    v = m3_build1 (ADDR_EXPR, t_addr, v);
    v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));
    v = m3_build1 (INDIRECT_REF, dst_t,
     m3_cast (build_pointer_type (dst_t), v));
  }
#endif
  TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */
  val = m3_cast (src_t, EXPR_REF (-1));
  if (src_T != dst_T) {
    val = m3_build1 (CONVERT_EXPR, dst_t, val);
  }
  add_stmt (build2 (MODIFY_EXPR, dst_t, v, val));
  EXPR_POP ();
}

m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T)
{
#if 0
  if (o != 0 || TREE_TYPE (v) != src_t) {
    v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),
     bitsize_int (o));
  }
#else
  /* failsafe, but inefficient */
  if (o != 0 || TREE_TYPE (v) != src_t) {
    v = m3_build1 (ADDR_EXPR, t_addr, v);
    v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));
    v = m3_build1 (INDIRECT_REF, src_t,
     m3_cast (build_pointer_type (src_t), v));
  }
#endif
  TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */
  if (src_T != dst_T) {
    v = m3_build1 (CONVERT_EXPR, dst_t, v);
  }
  EXPR_PUSH (v);
}

I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).

In an early version it seems that many loads/stores of global variables is wrong.

The first code affected is RTLinker_ExpandModuleTable.

PROCEDURE ExpandModuleTable () =
  CONST InitialTableSize = 500;
  VAR new_mods: ADDRESS;  n_bytes: INTEGER;
  BEGIN
    IF (modules = NIL) THEN
      (* first time... *)
      max_modules := InitialTableSize;
      modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr));
      IF (modules = NIL) THEN Cstdlib.abort (); END;
    ELSE
      n_bytes := max_modules * BYTESIZE (RT0.ModulePtr);
      new_mods := Cstdlib.malloc (n_bytes + n_bytes);
      IF (new_mods = NIL) THEN Cstdlib.abort (); END;
      EVAL Cstring.memcpy (new_mods, modules, n_bytes);
      Cstdlib.free (modules);
      modules := new_mods;
      INC (max_modules, max_modules);
    END;
  END ExpandModuleTable;

It should read back modules as NIL.
What happens is it reads back modules as not NIL,
and then max_modules should also be zero, but is "large", leading to malloc to fail,
and abort called.

The output of cm3cg -y is reasonable.
It indicates offset loads.

What is incorrect is that in many loads and stores, but not all, the
offsets are lost.

(923) begin_procedure
  procedure RTLinker__ExpandModuleTable
 RTLinker__ExpandModuleTable(924) set_source_line
  source line  208
(925) load_nil
(926) store
  store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb
(927) set_source_line
  source line  207
(928) set_source_line
  source line  210
(929) load_nil
(930) load
  m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xb

Here is the incorrect code:

 .globl RTLinker__ExpandModuleTable
 .stabd 46,0,0
 .stabn 68,0,206,.LM108
.LM108:
.LFBB8:
 .set nomips16
 .ent RTLinker__ExpandModuleTable
RTLinker__ExpandModuleTable:
 .frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0
 .mask 0xd0000000,-8
 .fmask 0x00000000,0
 daddiu $sp,$sp,-64
 sd $31,56($sp)
 sd $fp,48($sp)
 sd $28,40($sp)
 move $fp,$sp
 lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
 daddu $28,$28,$25
 daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
.LBB9:
 .stabn 68,0,208,.LM109
.LM109:
 sd $0,16($fp)
 .stabn 68,0,210,.LM110
.LM110:
 dla $2,MM_RTLinker
 ld $2,0($2)      << WRONG
 bne $2,$0,.L61
 .stabn 68,0,212,.LM111
.LM111:
 dla $3,MM_RTLinker
 ld $2,128($3)
 andi $2,$2,0x0
 ori $2,$2,0x1f4
 sd $2,128($3)
 .stabn 68,0,213,.LM112
.LM112:
 li $4,4000   # 0xfa0
 jal malloc
 sd $2,0($fp)
 ld $4,0($fp)
 dla $3,MM_RTLinker
 ld $2,136($3)
 andi $2,$2,0x0
 or $2,$4,$2
 sd $2,136($3)     << OK, offset seems low, but it agrees with elsewhere 
 .stabn 68,0,214,.LM113
.LM113:
 dla $2,MM_RTLinker
 ld $2,0($2)       << WRONG 
 bne $2,$0,.L65
 jal abort
 b .L65
.L61:
 .stabn 68,0,216,.LM114
.LM114:
 dla $2,MM_RTLinker
 ld $2,0($2)        << WRONG 
 dsll $2,$2,3
 sd $2,8($fp)
 .stabn 68,0,217,.LM115
.LM115:
 ld $3,8($fp)
 ld $2,8($fp)
 daddu $2,$3,$2
 move $4,$2
 jal malloc
 sd $2,0($fp)
 ld $2,0($fp)
 sd $2,16($fp)
 .stabn 68,0,218,.LM116
.LM116:
 ld $2,16($fp)
 bne $2,$0,.L64
 jal abort
.L64:
 .stabn 68,0,219,.LM117
.LM117:
 ld $4,16($fp)
 dla $2,MM_RTLinker
 ld $2,0($2)          << WRONG
 ld $3,8($fp)
 move $5,$2
 move $6,$3
 jal memcpy
 sd $2,0($fp)
 .stabn 68,0,220,.LM118
.LM118:
 dla $2,MM_RTLinker
 ld $2,0($2)           << WRONG
 move $4,$2
 jal free
 .stabn 68,0,221,.LM119
.LM119:
 ld $4,16($fp)
 dla $3,MM_RTLinker
 ld $2,136($3)         << OK
 andi $2,$2,0x0
 or $2,$4,$2
 sd $2,136($3)         << OK 
 .stabn 68,0,222,.LM120
.LM120:
 dla $2,MM_RTLinker
 ld $3,0($2)           << WRONG
 dla $2,MM_RTLinker
 ld $2,0($2)           << WRONG
 daddu $4,$3,$2
 dla $3,MM_RTLinker
 ld $2,128($3)         << OK
 andi $2,$2,0x0
 or $2,$2,$4
 sd $2,128($3)         << OK
.L65:
.LBE9:
 .stabn 68,0,224,.LM121
.LM121:
 move $sp,$fp
 ld $31,56($sp)
 ld $fp,48($sp)
 ld $28,40($sp)
 daddiu $sp,$sp,64
 j $31
 .end RTLinker__ExpandModuleTable

Here is corrected code:

RTLinker__ExpandModuleTable:
 .frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0
 .mask 0xd0000000,-8
 .fmask 0x00000000,0
 daddiu $sp,$sp,-64
 sd $31,56($sp)
 sd $fp,48($sp)
 sd $28,40($sp)
 move $fp,$sp
 lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
 daddu $28,$28,$25
 daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
.LBB9:
 .stabn 68,0,208,.LM109
.LM109:
 sd $0,16($fp)
 .stabn 68,0,210,.LM110
.LM110:
 dla $2,MM_RTLinker
 daddiu $2,$2,136     << CORRECT (offset seems low, but at least it isn't zero) 
 ld $2,0($2)
 bne $2,$0,.L61
 .stabn 68,0,212,.LM111
.LM111:
 dla $2,MM_RTLinker
 daddiu $3,$2,128     << CORRECT 
 li $2,500   # 0x1f4
 sd $2,0($3)
 .stabn 68,0,213,.LM112
.LM112:
 li $4,4000   # 0xfa0
 jal malloc
 sd $2,0($fp)
 dla $2,MM_RTLinker
 daddiu $3,$2,136      << CORRECT 
 ld $2,0($fp)
 sd $2,0($3)
 .stabn 68,0,214,.LM113
.LM113:
 dla $2,MM_RTLinker
 daddiu $2,$2,136      << CORRECT 
 ld $2,0($2)
 bne $2,$0,.L65
 jal abort
 b .L65
.L61:

I still dump core with this change, but I get much further.
I have not tested this change at all otherwise -- i.e. on working platforms.

I have not debugged through the gcc/m3cg code to see why
the offsets are only sometimes lost, or why other platforms
have no problem. Is MIPS's "global pointer" unusual these days?
I know SPARC has similar, and all the non-x86, non-AMD64 NT platforms
had similar, but that's essentially nothing now (I'm not counting IA64).
I know Linux has PLT (procedure linkage table) and GOT (global offset table).
Same thing?
I haven't looked much at other MIPS ports yet.

http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8

is what made it look about like it does -- in particularing using BIT_FIELD_REF.

1.7:

static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */
     tree v;
     int o;
     tree src_t, dest_t;
     m3_type src_T, dest_T;
{
  if (o == 0 && TREE_TYPE (v) == src_t) {
    EXPR_PUSH (v);
  } else {
    tree adr = m3_build1 (ADDR_EXPR, t_addr, v);
    if (o != 0) {
      adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT));
    }
    EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t,
                          m3_cast (build_pointer_type (src_t), adr)));
  }
#if 1
  if (src_T != dest_T) {
    EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1));
  }
  if (debug_vars) {
    const char *name = "noname";
    if (v != 0 && DECL_NAME(v) != 0) {
      name = IDENTIFIER_POINTER(DECL_NAME(v));
    }
    fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n", name,
            o, src_T, dest_T);
  }
#else
  if (src_T != dest_T) {
    if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) {
      EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1));
    } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) {
      EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1));
    } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) {
      /* okay */
    } else {
      fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n",
                  src_T, dest_T);
    }
  }
#endif
}
static void
m3_store (v, o, src_t, dest_t)  /* GCC32OK */
     tree v;
     int o;
     tree src_t, dest_t;
{
  tree lhs, rhs;
  if (TREE_TYPE (EXPR_REF (-1)) == src_t) {
    rhs = EXPR_REF (-1);
  } else {
    rhs = m3_cast (src_t, EXPR_REF (-1));
  }
  if (o == 0 && TREE_TYPE (v) == dest_t) {
    lhs = v;
  } else {
    tree f = make_node (FIELD_DECL);
    TREE_TYPE (f) = dest_t;
    DECL_ALIGN (f) = TYPE_ALIGN (dest_t);
    DECL_SIZE (f) = TYPE_SIZE (dest_t);
    DECL_MODE (f) = TYPE_MODE (dest_t);
    DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT);
    DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT);
    DECL_FIELD_CONTEXT (f) = TREE_TYPE (v);
    lhs = m3_build2 (COMPONENT_REF, dest_t, v, f);
  }
#if 1
    expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);
#else
  if (dest_t ==  NULL) {
    expand_assignment (lhs, rhs, 0, 0);
  } else {
    expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);
  }
#endif
  EXPR_POP ();
}

1.8:
static void
m3_load (tree v, int o,
  tree src_t, m3_type src_T,
  tree dst_t, m3_type dst_T) /* GCC32OK */
{
  if (o == 0 && TREE_TYPE (v) == src_t) {
    EXPR_PUSH (v);
  } else {
    EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),
     bitsize_int (o)));
  }
  if (src_T != dst_T) {
    EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1));
  }
  if (option_vars_trace) {
    const char *name = "noname";
    if (v != 0 && DECL_NAME(v) != 0) {
      name = IDENTIFIER_POINTER(DECL_NAME(v));
    }
    fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n", name,
     o, src_T, dst_T);
  }
}
static void
m3_store (tree v, int o, tree src_t, tree dst_t)  /* GCC32OK */
{
  tree lhs, rhs;
  if (TREE_TYPE (EXPR_REF (-1)) == src_t) {
    rhs = EXPR_REF (-1);
  } else {
    rhs = m3_cast (src_t, EXPR_REF (-1));
  }
  if (o == 0 && TREE_TYPE (v) == dst_t) {
    lhs = v;
  } else {
    lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),
       bitsize_int (o));
  }
  if (src_t != dst_t) {
    rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs);
  }
  expand_assignment (lhs, rhs, 0, 0);
  EXPR_POP ();
}

You should be able to repro this by configuring m3cg for mips64-openbsd.
I've only tested on cygwin host, but I could check others..
 
Thanks,
 - Jay

From jay.krell at cornell.edu  Sat Nov  8 16:45:15 2008
From: jay.krell at cornell.edu (Jay)
Date: Sat, 8 Nov 2008 15:45:15 +0000
Subject: [M3devel] FW: m3_load/m3_store/bit_field_ref lose offset??
Message-ID: <COL101-W3DB0BA622C4D31A82B602E6180@phx.gbl>


truncated as usual...along with losing a lot of whitespace..
Let's see...


Tony, how about switching these #if's?
I explain some of why below.

static void
m3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T)
{
  tree val;
#if 0
  if (o != 0 || TREE_TYPE (v) != dst_t) {
    v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),
     bitsize_int (o));
  }
#else
  /* failsafe, but inefficient */
  if (o != 0 || TREE_TYPE (v) != dst_t) {
    v = m3_build1 (ADDR_EXPR, t_addr, v);
    v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));
    v = m3_build1 (INDIRECT_REF, dst_t,
     m3_cast (build_pointer_type (dst_t), v));
  }
#endif
  TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */
  val = m3_cast (src_t, EXPR_REF (-1));
  if (src_T != dst_T) {
    val = m3_build1 (CONVERT_EXPR, dst_t, val);
  }
  add_stmt (build2 (MODIFY_EXPR, dst_t, v, val));
  EXPR_POP ();
}
m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T)
{
#if 0
  if (o != 0 || TREE_TYPE (v) != src_t) {
    v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),
     bitsize_int (o));
  }
#else
  /* failsafe, but inefficient */
  if (o != 0 || TREE_TYPE (v) != src_t) {
    v = m3_build1 (ADDR_EXPR, t_addr, v);
    v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));
    v = m3_build1 (INDIRECT_REF, src_t,
     m3_cast (build_pointer_type (src_t), v));
  }
#endif
  TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */
  if (src_T != dst_T) {
    v = m3_build1 (CONVERT_EXPR, dst_t, v);
  }
  EXPR_PUSH (v);
}

I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).
In an early version it seems that many loads/stores of global variables is wrong.
The "first" code affected is RTLinker_ExpandModuleTable.

PROCEDURE ExpandModuleTable () =
  CONST InitialTableSize = 500;
  VAR new_mods: ADDRESS;  n_bytes: INTEGER;
  BEGIN
    IF (modules = NIL) THEN
      (* first time... *)
      max_modules := InitialTableSize;
      modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr));
      IF (modules = NIL) THEN Cstdlib.abort (); END;
    ELSE
      n_bytes := max_modules * BYTESIZE (RT0.ModulePtr);
      new_mods := Cstdlib.malloc (n_bytes + n_bytes);
      IF (new_mods = NIL) THEN Cstdlib.abort (); END;
      EVAL Cstring.memcpy (new_mods, modules, n_bytes);
      Cstdlib.free (modules);
      modules := new_mods;
      INC (max_modules, max_modules);
    END;
  END ExpandModuleTable;

It should read back modules as NIL.
What happens is it reads back modules as not NIL,
and then max_modules should also be zero, but is "large", leading to malloc to fail,
and abort called.

The output of cm3cg -y is reasonable.
It indicates offset loads.

What is incorrect is that in many loads and stores, but not all, the
offsets are lost.

(923) begin_procedure
  procedure RTLinker__ExpandModuleTable
 RTLinker__ExpandModuleTable(924) set_source_line
  source line  208
(925) load_nil
(926) store
  store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb
(927) set_source_line
  source line  207
(928) set_source_line
  source line  210
(929) load_nil
(930) load
  m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xb

Here is the incorrect code:

 .globl RTLinker__ExpandModuleTable
 .stabd 46,0,0
 .stabn 68,0,206,.LM108
.LM108:
.LFBB8:
 .set nomips16
 .ent RTLinker__ExpandModuleTable
RTLinker__ExpandModuleTable:
 .frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0
 .mask 0xd0000000,-8
 .fmask 0x00000000,0
 daddiu $sp,$sp,-64
 sd $31,56($sp)
 sd $fp,48($sp)
 sd $28,40($sp)
 move $fp,$sp
 lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
 daddu $28,$28,$25
 daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
.LBB9:
 .stabn 68,0,208,.LM109
.LM109:
 sd $0,16($fp)
 .stabn 68,0,210,.LM110
.LM110:
 dla $2,MM_RTLinker
 ld $2,0($2)      << WRONG
 bne $2,$0,.L61
 .stabn 68,0,212,.LM111
.LM111:
 dla $3,MM_RTLinker
 ld $2,128($3)
 andi $2,$2,0x0
 ori $2,$2,0x1f4
 sd $2,128($3)
 .stabn 68,0,213,.LM112
.LM112:
 li $4,4000   # 0xfa0
 jal malloc
 sd $2,0($fp)
 ld $4,0($fp)
 dla $3,MM_RTLinker
 ld $2,136($3)
 andi $2,$2,0x0
 or $2,$4,$2
 sd $2,136($3)     << OK, offset seems low, but it agrees with elsewhere 
 .stabn 68,0,214,.LM113
.LM113:
 dla $2,MM_RTLinker
 ld $2,0($2)       << WRONG 
 bne $2,$0,.L65
 jal abort
 b .L65
.L61:
 .stabn 68,0,216,.LM114
.LM114:
 dla $2,MM_RTLinker
 ld $2,0($2)        << WRONG 
 dsll $2,$2,3
 sd $2,8($fp)
 .stabn 68,0,217,.LM115
.LM115:
 ld $3,8($fp)
 ld $2,8($fp)
 daddu $2,$3,$2
 move $4,$2
 jal malloc
 sd $2,0($fp)
 ld $2,0($fp)
 sd $2,16($fp)
 .stabn 68,0,218,.LM116
.LM116:
 ld $2,16($fp)
 bne $2,$0,.L64
 jal abort
.L64:
 .stabn 68,0,219,.LM117
.LM117:
 ld $4,16($fp)
 dla $2,MM_RTLinker
 ld $2,0($2)          << WRONG
 ld $3,8($fp)
 move $5,$2
 move $6,$3
 jal memcpy
 sd $2,0($fp)
 .stabn 68,0,220,.LM118
.LM118:
 dla $2,MM_RTLinker
 ld $2,0($2)           << WRONG
 move $4,$2
 jal free
 .stabn 68,0,221,.LM119
.LM119:
 ld $4,16($fp)
 dla $3,MM_RTLinker
 ld $2,136($3)         << OK
 andi $2,$2,0x0
 or $2,$4,$2
 sd $2,136($3)         << OK 
 .stabn 68,0,222,.LM120
.LM120:
 dla $2,MM_RTLinker
 ld $3,0($2)           << WRONG
 dla $2,MM_RTLinker
 ld $2,0($2)           << WRONG
 daddu $4,$3,$2
 dla $3,MM_RTLinker
 ld $2,128($3)         << OK
 andi $2,$2,0x0
 or $2,$2,$4
 sd $2,128($3)         << OK
.L65:
.LBE9:
 .stabn 68,0,224,.LM121
.LM121:
 move $sp,$fp
 ld $31,56($sp)
 ld $fp,48($sp)
 ld $28,40($sp)
 daddiu $sp,$sp,64
 j $31
 .end RTLinker__ExpandModuleTable

Here is corrected code:

RTLinker__ExpandModuleTable:
 .frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0
 .mask 0xd0000000,-8
 .fmask 0x00000000,0
 daddiu $sp,$sp,-64
 sd $31,56($sp)
 sd $fp,48($sp)
 sd $28,40($sp)
 move $fp,$sp
 lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
 daddu $28,$28,$25
 daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
.LBB9:
 .stabn 68,0,208,.LM109
.LM109:
 sd $0,16($fp)
 .stabn 68,0,210,.LM110
.LM110:
 dla $2,MM_RTLinker
 daddiu $2,$2,136     << CORRECT (offset seems low, but at least it isn't zero) 
 ld $2,0($2)
 bne $2,$0,.L61
 .stabn 68,0,212,.LM111
.LM111:
 dla $2,MM_RTLinker
 daddiu $3,$2,128     << CORRECT 
 li $2,500   # 0x1f4
 sd $2,0($3)
 .stabn 68,0,213,.LM112
.LM112:
 li $4,4000   # 0xfa0
 jal malloc
 sd $2,0($fp)
 dla $2,MM_RTLinker
 daddiu $3,$2,136      << CORRECT 
 ld $2,0($fp)
 sd $2,0($3)
 .stabn 68,0,214,.LM113
.LM113:
 dla $2,MM_RTLinker
 daddiu $2,$2,136      << CORRECT 
 ld $2,0($2)
 bne $2,$0,.L65
 jal abort
 b .L65
.L61:

I still dump core with this change, but I get much further.
I have not tested this change at all otherwise -- i.e. on working platforms.

I have not debugged through the gcc/m3cg code to see why
the offsets are only sometimes lost, or why other platforms
have no problem. Is MIPS's "global pointer" unusual these days?
I know SPARC has similar, and all the non-x86, non-AMD64 NT platforms
had similar, but that's essentially nothing now (I'm not counting IA64).
I know Linux has PLT (procedure linkage table) and GOT (global offset table).
Same thing?
I haven't looked much at other MIPS ports yet.

http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8

is what made it look about like it does -- in particularing using BIT_FIELD_REF.

1.7:

static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */
     tree v;
     int o;
     tree src_t, dest_t;
     m3_type src_T, dest_T;
{
  if (o == 0 && TREE_TYPE (v) == src_t) {
    EXPR_PUSH (v);
  } else {
    tree adr = m3_build1 (ADDR_EXPR, t_addr, v);
    if (o != 0) {
      adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT));
    }
    EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t,
                          m3_cast (build_pointer_type (src_t), adr)));
  }
#if 1
  if (src_T != dest_T) {
    EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1));
  }
  if (debug_vars) {
    const char *name = "noname";
    if (v != 0 && DECL_NAME(v) != 0) {
      name = IDENTIFIER_POINTER(DECL_NAME(v));
    }
    fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n", name,
            o, src_T, dest_T);
  }
#else
  if (src_T != dest_T) {
    if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) {
      EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1));
    } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) {
      EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1));
    } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) {
      /* okay */
    } else {
      fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n",
                  src_T, dest_T);
    }
  }
#endif
}
static void
m3_store (v, o, src_t, dest_t)  /* GCC32OK */
     tree v;
     int o;
     tree src_t, dest_t;
{
  tree lhs, rhs;
  if (TREE_TYPE (EXPR_REF (-1)) == src_t) {
    rhs = EXPR_REF (-1);
  } else {
    rhs = m3_cast (src_t, EXPR_REF (-1));
  }
  if (o == 0 && TREE_TYPE (v) == dest_t) {
    lhs = v;
  } else {
    tree f = make_node (FIELD_DECL);
    TREE_TYPE (f) = dest_t;
    DECL_ALIGN (f) = TYPE_ALIGN (dest_t);
    DECL_SIZE (f) = TYPE_SIZE (dest_t);
    DECL_MODE (f) = TYPE_MODE (dest_t);
    DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT);
    DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT);
    DECL_FIELD_CONTEXT (f) = TREE_TYPE (v);
    lhs = m3_build2 (COMPONENT_REF, dest_t, v, f);
  }
#if 1
    expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);
#else
  if (dest_t ==  NULL) {
    expand_assignment (lhs, rhs, 0, 0);
  } else {
    expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);
  }
#endif
  EXPR_POP ();
}

1.8:
static void
m3_load (tree v, int o,
  tree src_t, m3_type src_T,
  tree dst_t, m3_type dst_T) /* GCC32OK */
{
  if (o == 0 && TREE_TYPE (v) == src_t) {
    EXPR_PUSH (v);
  } else {
    EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),
     bitsize_int (o)));
  }
  if (src_T != dst_T) {
    EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1));
  }
  if (option_vars_trace) {
    const char *name = "noname";
    if (v != 0 && DECL_NAME(v) != 0) {
      name = IDENTIFIER_POINTER(DECL_NAME(v));
    }
    fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n", name,
     o, src_T, dst_T);
  }
}
static void
m3_store (tree v, int o, tree src_t, tree dst_t)  /* GCC32OK */
{
  tree lhs, rhs;
  if (TREE_TYPE (EXPR_REF (-1)) == src_t) {
    rhs = EXPR_REF (-1);
  } else {
    rhs = m3_cast (src_t, EXPR_REF (-1));
  }
  if (o == 0 && TREE_TYPE (v) == dst_t) {
    lhs = v;
  } else {
    lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),
       bitsize_int (o));
  }
  if (src_t != dst_t) {
    rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs);
  }
  expand_assignment (lhs, rhs, 0, 0);
  EXPR_POP ();
}

 - Jay

From hosking at cs.purdue.edu  Sat Nov  8 23:55:54 2008
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Sat, 8 Nov 2008 22:55:54 +0000
Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
In-Reply-To: <COL101-W3DB0BA622C4D31A82B602E6180@phx.gbl>
References: <COL101-W3DB0BA622C4D31A82B602E6180@phx.gbl>
Message-ID: <C47F6B5A-F154-4485-B839-96185B59E3B6@cs.purdue.edu>

Have you checked to make sure that the pre-gcc IR makes sense?  Does  
it see these values as 32-bit or 64-bit?  The reason I ask is that  
there may be something wrong in your m3middle target specs that cause  
bad IR.  Once we know the IR looks right then we can try and fix the  
problem.

Antony Hosking | Associate Professor | Computer Science | Purdue  
University
305 N. University Street | West Lafayette | IN 47907 | USA
Office +1 765 494 6001 | Mobile +1 765 427 5484



On 8 Nov 2008, at 15:45, Jay wrote:

>
> truncated as usual...along with losing a lot of whitespace..
> Let's see...
>
>
> Tony, how about switching these #if's?
> I explain some of why below.
>
> static void
> m3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t,  
> m3_type dst_T)
> {
>  tree val;
> #if 0
>  if (o != 0 || TREE_TYPE (v) != dst_t) {
>    v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),
>     bitsize_int (o));
>  }
> #else
>  /* failsafe, but inefficient */
>  if (o != 0 || TREE_TYPE (v) != dst_t) {
>    v = m3_build1 (ADDR_EXPR, t_addr, v);
>    v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));
>    v = m3_build1 (INDIRECT_REF, dst_t,
>     m3_cast (build_pointer_type (dst_t), v));
>  }
> #endif
>  TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing  
> problems */
>  val = m3_cast (src_t, EXPR_REF (-1));
>  if (src_T != dst_T) {
>    val = m3_build1 (CONVERT_EXPR, dst_t, val);
>  }
>  add_stmt (build2 (MODIFY_EXPR, dst_t, v, val));
>  EXPR_POP ();
> }
> m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t,  
> m3_type dst_T)
> {
> #if 0
>  if (o != 0 || TREE_TYPE (v) != src_t) {
>    v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),
>     bitsize_int (o));
>  }
> #else
>  /* failsafe, but inefficient */
>  if (o != 0 || TREE_TYPE (v) != src_t) {
>    v = m3_build1 (ADDR_EXPR, t_addr, v);
>    v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));
>    v = m3_build1 (INDIRECT_REF, src_t,
>     m3_cast (build_pointer_type (src_t), v));
>  }
> #endif
>  TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing  
> problems */
>  if (src_T != dst_T) {
>    v = m3_build1 (CONVERT_EXPR, dst_t, v);
>  }
>  EXPR_PUSH (v);
> }
>
> I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).
> In an early version it seems that many loads/stores of global  
> variables is wrong.
> The "first" code affected is RTLinker_ExpandModuleTable.
>
> PROCEDURE ExpandModuleTable () =
>  CONST InitialTableSize = 500;
>  VAR new_mods: ADDRESS;  n_bytes: INTEGER;
>  BEGIN
>    IF (modules = NIL) THEN
>      (* first time... *)
>      max_modules := InitialTableSize;
>      modules := Cstdlib.malloc (InitialTableSize * BYTESIZE  
> (RT0.ModulePtr));
>      IF (modules = NIL) THEN Cstdlib.abort (); END;
>    ELSE
>      n_bytes := max_modules * BYTESIZE (RT0.ModulePtr);
>      new_mods := Cstdlib.malloc (n_bytes + n_bytes);
>      IF (new_mods = NIL) THEN Cstdlib.abort (); END;
>      EVAL Cstring.memcpy (new_mods, modules, n_bytes);
>      Cstdlib.free (modules);
>      modules := new_mods;
>      INC (max_modules, max_modules);
>    END;
>  END ExpandModuleTable;
>
> It should read back modules as NIL.
> What happens is it reads back modules as not NIL,
> and then max_modules should also be zero, but is "large", leading to  
> malloc to fail,
> and abort called.
>
> The output of cm3cg -y is reasonable.
> It indicates offset loads.
>
> What is incorrect is that in many loads and stores, but not all, the
> offsets are lost.
>
> (923) begin_procedure
>  procedure RTLinker__ExpandModuleTable
> RTLinker__ExpandModuleTable(924) set_source_line
>  source line  208
> (925) load_nil
> (926) store
>  store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb
> (927) set_source_line
>  source line  207
> (928) set_source_line
>  source line  210
> (929) load_nil
> (930) load
>  m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xb
>
> Here is the incorrect code:
>
> .globl RTLinker__ExpandModuleTable
> .stabd 46,0,0
> .stabn 68,0,206,.LM108
> .LM108:
> .LFBB8:
> .set nomips16
> .ent RTLinker__ExpandModuleTable
> RTLinker__ExpandModuleTable:
> .frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0
> .mask 0xd0000000,-8
> .fmask 0x00000000,0
> daddiu $sp,$sp,-64
> sd $31,56($sp)
> sd $fp,48($sp)
> sd $28,40($sp)
> move $fp,$sp
> lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
> daddu $28,$28,$25
> daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
> .LBB9:
> .stabn 68,0,208,.LM109
> .LM109:
> sd $0,16($fp)
> .stabn 68,0,210,.LM110
> .LM110:
> dla $2,MM_RTLinker
> ld $2,0($2)      << WRONG
> bne $2,$0,.L61
> .stabn 68,0,212,.LM111
> .LM111:
> dla $3,MM_RTLinker
> ld $2,128($3)
> andi $2,$2,0x0
> ori $2,$2,0x1f4
> sd $2,128($3)
> .stabn 68,0,213,.LM112
> .LM112:
> li $4,4000   # 0xfa0
> jal malloc
> sd $2,0($fp)
> ld $4,0($fp)
> dla $3,MM_RTLinker
> ld $2,136($3)
> andi $2,$2,0x0
> or $2,$4,$2
> sd $2,136($3)     << OK, offset seems low, but it agrees with  
> elsewhere
> .stabn 68,0,214,.LM113
> .LM113:
> dla $2,MM_RTLinker
> ld $2,0($2)       << WRONG
> bne $2,$0,.L65
> jal abort
> b .L65
> .L61:
> .stabn 68,0,216,.LM114
> .LM114:
> dla $2,MM_RTLinker
> ld $2,0($2)        << WRONG
> dsll $2,$2,3
> sd $2,8($fp)
> .stabn 68,0,217,.LM115
> .LM115:
> ld $3,8($fp)
> ld $2,8($fp)
> daddu $2,$3,$2
> move $4,$2
> jal malloc
> sd $2,0($fp)
> ld $2,0($fp)
> sd $2,16($fp)
> .stabn 68,0,218,.LM116
> .LM116:
> ld $2,16($fp)
> bne $2,$0,.L64
> jal abort
> .L64:
> .stabn 68,0,219,.LM117
> .LM117:
> ld $4,16($fp)
> dla $2,MM_RTLinker
> ld $2,0($2)          << WRONG
> ld $3,8($fp)
> move $5,$2
> move $6,$3
> jal memcpy
> sd $2,0($fp)
> .stabn 68,0,220,.LM118
> .LM118:
> dla $2,MM_RTLinker
> ld $2,0($2)           << WRONG
> move $4,$2
> jal free
> .stabn 68,0,221,.LM119
> .LM119:
> ld $4,16($fp)
> dla $3,MM_RTLinker
> ld $2,136($3)         << OK
> andi $2,$2,0x0
> or $2,$4,$2
> sd $2,136($3)         << OK
> .stabn 68,0,222,.LM120
> .LM120:
> dla $2,MM_RTLinker
> ld $3,0($2)           << WRONG
> dla $2,MM_RTLinker
> ld $2,0($2)           << WRONG
> daddu $4,$3,$2
> dla $3,MM_RTLinker
> ld $2,128($3)         << OK
> andi $2,$2,0x0
> or $2,$2,$4
> sd $2,128($3)         << OK
> .L65:
> .LBE9:
> .stabn 68,0,224,.LM121
> .LM121:
> move $sp,$fp
> ld $31,56($sp)
> ld $fp,48($sp)
> ld $28,40($sp)
> daddiu $sp,$sp,64
> j $31
> .end RTLinker__ExpandModuleTable
>
> Here is corrected code:
>
> RTLinker__ExpandModuleTable:
> .frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0
> .mask 0xd0000000,-8
> .fmask 0x00000000,0
> daddiu $sp,$sp,-64
> sd $31,56($sp)
> sd $fp,48($sp)
> sd $28,40($sp)
> move $fp,$sp
> lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
> daddu $28,$28,$25
> daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
> .LBB9:
> .stabn 68,0,208,.LM109
> .LM109:
> sd $0,16($fp)
> .stabn 68,0,210,.LM110
> .LM110:
> dla $2,MM_RTLinker
> daddiu $2,$2,136     << CORRECT (offset seems low, but at least it  
> isn't zero)
> ld $2,0($2)
> bne $2,$0,.L61
> .stabn 68,0,212,.LM111
> .LM111:
> dla $2,MM_RTLinker
> daddiu $3,$2,128     << CORRECT
> li $2,500   # 0x1f4
> sd $2,0($3)
> .stabn 68,0,213,.LM112
> .LM112:
> li $4,4000   # 0xfa0
> jal malloc
> sd $2,0($fp)
> dla $2,MM_RTLinker
> daddiu $3,$2,136      << CORRECT
> ld $2,0($fp)
> sd $2,0($3)
> .stabn 68,0,214,.LM113
> .LM113:
> dla $2,MM_RTLinker
> daddiu $2,$2,136      << CORRECT
> ld $2,0($2)
> bne $2,$0,.L65
> jal abort
> b .L65
> .L61:
>
> I still dump core with this change, but I get much further.
> I have not tested this change at all otherwise -- i.e. on working  
> platforms.
>
> I have not debugged through the gcc/m3cg code to see why
> the offsets are only sometimes lost, or why other platforms
> have no problem. Is MIPS's "global pointer" unusual these days?
> I know SPARC has similar, and all the non-x86, non-AMD64 NT platforms
> had similar, but that's essentially nothing now (I'm not counting  
> IA64).
> I know Linux has PLT (procedure linkage table) and GOT (global  
> offset table).
> Same thing?
> I haven't looked much at other MIPS ports yet.
>
> http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8
>
> is what made it look about like it does -- in particularing using  
> BIT_FIELD_REF.
>
> 1.7:
>
> static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */
>     tree v;
>     int o;
>     tree src_t, dest_t;
>     m3_type src_T, dest_T;
> {
>  if (o == 0 && TREE_TYPE (v) == src_t) {
>    EXPR_PUSH (v);
>  } else {
>    tree adr = m3_build1 (ADDR_EXPR, t_addr, v);
>    if (o != 0) {
>      adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o /  
> BITS_PER_UNIT));
>    }
>    EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t,
>                          m3_cast (build_pointer_type (src_t), adr)));
>  }
> #if 1
>  if (src_T != dest_T) {
>    EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1));
>  }
>  if (debug_vars) {
>    const char *name = "noname";
>    if (v != 0 && DECL_NAME(v) != 0) {
>      name = IDENTIFIER_POINTER(DECL_NAME(v));
>    }
>    fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n",  
> name,
>            o, src_T, dest_T);
>  }
> #else
>  if (src_T != dest_T) {
>    if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) {
>      EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1));
>    } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) {
>      EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1));
>    } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) {
>      /* okay */
>    } else {
>      fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n",
>                  src_T, dest_T);
>    }
>  }
> #endif
> }
> static void
> m3_store (v, o, src_t, dest_t)  /* GCC32OK */
>     tree v;
>     int o;
>     tree src_t, dest_t;
> {
>  tree lhs, rhs;
>  if (TREE_TYPE (EXPR_REF (-1)) == src_t) {
>    rhs = EXPR_REF (-1);
>  } else {
>    rhs = m3_cast (src_t, EXPR_REF (-1));
>  }
>  if (o == 0 && TREE_TYPE (v) == dest_t) {
>    lhs = v;
>  } else {
>    tree f = make_node (FIELD_DECL);
>    TREE_TYPE (f) = dest_t;
>    DECL_ALIGN (f) = TYPE_ALIGN (dest_t);
>    DECL_SIZE (f) = TYPE_SIZE (dest_t);
>    DECL_MODE (f) = TYPE_MODE (dest_t);
>    DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT);
>    DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT);
>    DECL_FIELD_CONTEXT (f) = TREE_TYPE (v);
>    lhs = m3_build2 (COMPONENT_REF, dest_t, v, f);
>  }
> #if 1
>    expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0,  
> 0);
> #else
>  if (dest_t ==  NULL) {
>    expand_assignment (lhs, rhs, 0, 0);
>  } else {
>    expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0,  
> 0);
>  }
> #endif
>  EXPR_POP ();
> }
>
> 1.8:
> static void
> m3_load (tree v, int o,
>  tree src_t, m3_type src_T,
>  tree dst_t, m3_type dst_T) /* GCC32OK */
> {
>  if (o == 0 && TREE_TYPE (v) == src_t) {
>    EXPR_PUSH (v);
>  } else {
>    EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),
>     bitsize_int (o)));
>  }
>  if (src_T != dst_T) {
>    EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1));
>  }
>  if (option_vars_trace) {
>    const char *name = "noname";
>    if (v != 0 && DECL_NAME(v) != 0) {
>      name = IDENTIFIER_POINTER(DECL_NAME(v));
>    }
>    fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n",  
> name,
>     o, src_T, dst_T);
>  }
> }
> static void
> m3_store (tree v, int o, tree src_t, tree dst_t)  /* GCC32OK */
> {
>  tree lhs, rhs;
>  if (TREE_TYPE (EXPR_REF (-1)) == src_t) {
>    rhs = EXPR_REF (-1);
>  } else {
>    rhs = m3_cast (src_t, EXPR_REF (-1));
>  }
>  if (o == 0 && TREE_TYPE (v) == dst_t) {
>    lhs = v;
>  } else {
>    lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),
>       bitsize_int (o));
>  }
>  if (src_t != dst_t) {
>    rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs);
>  }
>  expand_assignment (lhs, rhs, 0, 0);
>  EXPR_POP ();
> }
>
> - Jay

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081108/4bd834d6/attachment.html>

From jay.krell at cornell.edu  Sun Nov  9 02:25:32 2008
From: jay.krell at cornell.edu (Jay)
Date: Sun, 9 Nov 2008 01:25:32 +0000
Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
In-Reply-To: <C47F6B5A-F154-4485-B839-96185B59E3B6@cs.purdue.edu>
References: <COL101-W3DB0BA622C4D31A82B602E6180@phx.gbl>
	<C47F6B5A-F154-4485-B839-96185B59E3B6@cs.purdue.edu>
Message-ID: <COL101-W2CFF8666D1486BD514159E61B0@phx.gbl>


Thanks for the tip/direction.
 
Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the same" but for two systematic diffs:
 
 - AMD64_LINUX has a bunch of added calls to check_nil.
   This is a "bug" in it, inoptimal but ok code, because "first readable address" is 0. It should probably be 4k. I should write a program that iterates starting at 0 to find the value.
  (or better yet, cm3 should have a -configure option that does this? -- for configuration values for which there is a safe default; in general, imho, the targets should be auto configured via C, assuming there is a native development environment available).
 
 - some integers are different (again, systematically) 
 
AMD64_LINUX:
 init_int  656 13 Int.64 init_chars  664 "    typecode " init_int  680 2 Int.64 init_var  688 v.1 0 init_int  696 13 Int.64 init_chars  704 "    typeid   " init_int  720 2 Int.64 init_var  728 v.1 0 init_int  736 13 Int.64 init_chars  744 "RunMainBody: " init_int  760 2 Int.64 init_var  768 v.1 0 init_int  776 19 Int.64 init_chars  784 "RunMainBody: exec: " init_int  808 2 Int.64 init_var  816 v.1 0 init_int  824 17 Int.64 init_chars  832 "ExpandInitStack: "
MIPS64_OPENBSD:
 
 init_chars  664 "    typecode " init_int  680 8796093022208 Int.64 init_var  688 v.1 0 init_int  696 13 Int.64 init_chars  704 "    typeid   " init_int  720 8796093022208 Int.64 init_var  728 v.1 0 init_int  736 13 Int.64 init_chars  744 "RunMainBody: " init_int  760 8796093022208 Int.64 init_var  768 v.1 0 init_int  776 19 Int.64 init_chars  784 "RunMainBody: exec: " init_int  808 8796093022208 Int.64 init_var  816 v.1 0This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where little endian has 2.
 
I don't know what these numbers represent.
I should look at other systems -- preferably a working 64 bit big endian system, if there are any.
 
Could be flags/type with text literals?
 
(Btw, I always leave the TextLiteral.i3 change on my system, lowering the maximum literal length to around 4gig even on 64 bit targets, so 32 bit can bootstrap 64 bit...need to fix the compiler here to do target arithmetic instead of host arithmetic...)
 
Thanks, - Jay



From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sat, 8 Nov 2008 22:55:54 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Have you checked to make sure that the pre-gcc IR makes sense?  Does it see these values as 32-bit or 64-bit?  The reason I ask is that there may be something wrong in your m3middle target specs that cause bad IR.  Once we know the IR looks right then we can try and fix the problem.



Antony Hosking | Associate Professor | Computer Science | Purdue University
305 N. University Street | West Lafayette | IN 47907 | USA
Office +1 765 494 6001 | Mobile +1 765 427 5484


On 8 Nov 2008, at 15:45, Jay wrote:

truncated as usual...along with losing a lot of whitespace..Let's see...Tony, how about switching these #if's?I explain some of why below.static voidm3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){ tree val;#if 0 if (o != 0 || TREE_TYPE (v) != dst_t) {   v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),    bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) {   v = m3_build1 (ADDR_EXPR, t_addr, v);   v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));   v = m3_build1 (INDIRECT_REF, dst_t,    m3_cast (build_pointer_type (dst_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) {   val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP ();}m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){#if 0 if (o != 0 || TREE_TYPE (v) != src_t) {   v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),    bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) {   v = m3_build1 (ADDR_EXPR, t_addr, v);   v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));   v = m3_build1 (INDIRECT_REF, src_t,    m3_cast (build_pointer_type (src_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) {   v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v);}I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).In an early version it seems that many loads/stores of global variables is wrong.The "first" code affected is RTLinker_ExpandModuleTable.PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS;  n_bytes: INTEGER; BEGIN   IF (modules = NIL) THEN     (* first time... *)     max_modules := InitialTableSize;     modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr));     IF (modules = NIL) THEN Cstdlib.abort (); END;   ELSE     n_bytes := max_modules * BYTESIZE (RT0.ModulePtr);     new_mods := Cstdlib.malloc (n_bytes + n_bytes);     IF (new_mods = NIL) THEN Cstdlib.abort (); END;     EVAL Cstring.memcpy (new_mods, modules, n_bytes);     Cstdlib.free (modules);     modules := new_mods;     INC (max_modules, max_modules);   END; END ExpandModuleTable;It should read back modules as NIL.What happens is it reads back modules as not NIL,and then max_modules should also be zero, but is "large", leading to malloc to fail,and abort called.The output of cm3cg -y is reasonable.It indicates offset loads.What is incorrect is that in many loads and stores, but not all, theoffsets are lost.(923) begin_procedure procedure RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable(924) set_source_line source line  208(925) load_nil(926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb(927) set_source_line source line  207(928) set_source_line source line  210(929) load_nil(930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xbHere is the incorrect code:.globl RTLinker__ExpandModuleTable.stabd 46,0,0.stabn 68,0,206,.LM108.LM108:.LFBB8:.set nomips16.ent RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable:.frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerld $2,0($2)      << WRONGbne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $3,MM_RTLinkerld $2,128($3)andi $2,$2,0x0ori $2,$2,0x1f4sd $2,128($3).stabn 68,0,213,.LM112.LM112:li $4,4000   # 0xfa0jal mallocsd $2,0($fp)ld $4,0($fp)dla $3,MM_RTLinkerld $2,136($3)andi $2,$2,0x0or $2,$4,$2sd $2,136($3)     << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerld $2,0($2)       << WRONG bne $2,$0,.L65jal abortb .L65.L61:.stabn 68,0,216,.LM114.LM114:dla $2,MM_RTLinkerld $2,0($2)        << WRONG dsll $2,$2,3sd $2,8($fp).stabn 68,0,217,.LM115.LM115:ld $3,8($fp)ld $2,8($fp)daddu $2,$3,$2move $4,$2jal mallocsd $2,0($fp)ld $2,0($fp)sd $2,16($fp).stabn 68,0,218,.LM116.LM116:ld $2,16($fp)bne $2,$0,.L64jal abort.L64:.stabn 68,0,219,.LM117.LM117:ld $4,16($fp)dla $2,MM_RTLinkerld $2,0($2)          << WRONGld $3,8($fp)move $5,$2move $6,$3jal memcpysd $2,0($fp).stabn 68,0,220,.LM118.LM118:dla $2,MM_RTLinkerld $2,0($2)           << WRONGmove $4,$2jal free.stabn 68,0,221,.LM119.LM119:ld $4,16($fp)dla $3,MM_RTLinkerld $2,136($3)         << OKandi $2,$2,0x0or $2,$4,$2sd $2,136($3)         << OK .stabn 68,0,222,.LM120.LM120:dla $2,MM_RTLinkerld $3,0($2)           << WRONGdla $2,MM_RTLinkerld $2,0($2)           << WRONGdaddu $4,$3,$2dla $3,MM_RTLinkerld $2,128($3)         << OKandi $2,$2,0x0or $2,$2,$4sd $2,128($3)         << OK.L65:.LBE9:.stabn 68,0,224,.LM121.LM121:move $sp,$fpld $31,56($sp)ld $fp,48($sp)ld $28,40($sp)daddiu $sp,$sp,64j $31.end RTLinker__ExpandModuleTableHere is corrected code:RTLinker__ExpandModuleTable:.frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerdaddiu $2,$2,136     << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2)bne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $2,MM_RTLinkerdaddiu $3,$2,128     << CORRECT li $2,500   # 0x1f4sd $2,0($3).stabn 68,0,213,.LM112.LM112:li $4,4000   # 0xfa0jal mallocsd $2,0($fp)dla $2,MM_RTLinkerdaddiu $3,$2,136      << CORRECT ld $2,0($fp)sd $2,0($3).stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerdaddiu $2,$2,136      << CORRECT ld $2,0($2)bne $2,$0,.L65jal abortb .L65.L61:I still dump core with this change, but I get much further.I have not tested this change at all otherwise -- i.e. on working platforms.I have not debugged through the gcc/m3cg code to see whythe offsets are only sometimes lost, or why other platformshave no problem. Is MIPS's "global pointer" unusual these days?I know SPARC has similar, and all the non-x86, non-AMD64 NT platformshad similar, but that's essentially nothing now (I'm not counting IA64).I know Linux has PLT (procedure linkage table) and GOT (global offset table).Same thing?I haven't looked much at other MIPS ports yet.http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8is what made it look about like it does -- in particularing using BIT_FIELD_REF.1.7:static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */    tree v;    int o;    tree src_t, dest_t;    m3_type src_T, dest_T;{ if (o == 0 && TREE_TYPE (v) == src_t) {   EXPR_PUSH (v); } else {   tree adr = m3_build1 (ADDR_EXPR, t_addr, v);   if (o != 0) {     adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT));   }   EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t,                         m3_cast (build_pointer_type (src_t), adr))); }#if 1 if (src_T != dest_T) {   EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) {   const char *name = "noname";   if (v != 0 && DECL_NAME(v) != 0) {     name = IDENTIFIER_POINTER(DECL_NAME(v));   }   fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n", name,           o, src_T, dest_T); }#else if (src_T != dest_T) {   if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) {     EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1));   } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) {     EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1));   } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) {     /* okay */   } else {     fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n",                 src_T, dest_T);   } }#endif}static voidm3_store (v, o, src_t, dest_t)  /* GCC32OK */    tree v;    int o;    tree src_t, dest_t;{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) {   rhs = EXPR_REF (-1); } else {   rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) {   lhs = v; } else {   tree f = make_node (FIELD_DECL);   TREE_TYPE (f) = dest_t;   DECL_ALIGN (f) = TYPE_ALIGN (dest_t);   DECL_SIZE (f) = TYPE_SIZE (dest_t);   DECL_MODE (f) = TYPE_MODE (dest_t);   DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT);   DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT);   DECL_FIELD_CONTEXT (f) = TREE_TYPE (v);   lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); }#if 1   expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);#else if (dest_t ==  NULL) {   expand_assignment (lhs, rhs, 0, 0); } else {   expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); }#endif EXPR_POP ();}1.8:static voidm3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */{ if (o == 0 && TREE_TYPE (v) == src_t) {   EXPR_PUSH (v); } else {   EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),    bitsize_int (o))); } if (src_T != dst_T) {   EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) {   const char *name = "noname";   if (v != 0 && DECL_NAME(v) != 0) {     name = IDENTIFIER_POINTER(DECL_NAME(v));   }   fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n", name,    o, src_T, dst_T); }}static voidm3_store (tree v, int o, tree src_t, tree dst_t)  /* GCC32OK */{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) {   rhs = EXPR_REF (-1); } else {   rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) {   lhs = v; } else {   lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),      bitsize_int (o)); } if (src_t != dst_t) {   rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP ();}- Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081109/2528e9ca/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: AMD64_LINUX_RTLinker.mc
Type: application/octet-stream
Size: 17622 bytes
Desc: not available
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081109/2528e9ca/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: MIPS64_OPENBSD_RTLinker.mc
Type: application/octet-stream
Size: 17692 bytes
Desc: not available
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081109/2528e9ca/attachment-0001.obj>

From hosking at cs.purdue.edu  Sun Nov  9 11:01:58 2008
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Sun, 9 Nov 2008 10:01:58 +0000
Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
In-Reply-To: <COL101-W2CFF8666D1486BD514159E61B0@phx.gbl>
References: <COL101-W3DB0BA622C4D31A82B602E6180@phx.gbl>
	<C47F6B5A-F154-4485-B839-96185B59E3B6@cs.purdue.edu>
	<COL101-W2CFF8666D1486BD514159E61B0@phx.gbl>
Message-ID: <B9A95C1E-B873-40A9-A045-CD38E32CA5D0@cs.purdue.edu>

Aren't the integer values supposed to be written endian-independently  
in the IR?  I am suspicious about these differences.

On 9 Nov 2008, at 01:25, Jay wrote:

> Thanks for the tip/direction.
>
> Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the  
> same" but for two systematic diffs:
>
>  - AMD64_LINUX has a bunch of added calls to check_nil.
>    This is a "bug" in it, inoptimal but ok code, because "first  
> readable address" is 0. It should probably be 4k. I should write a  
> program that iterates starting at 0 to find the value.
>   (or better yet, cm3 should have a -configure option that does  
> this? -- for configuration values for which there is a safe default;  
> in general, imho, the targets should be auto configured via C,  
> assuming there is a native development environment available).
>
>  - some integers are different (again, systematically)
>
> AMD64_LINUX:
>  init_int  656 13 Int.64
>  init_chars  664 "    typecode "
>  init_int  680 2 Int.64
>  init_var  688 v.1 0
>  init_int  696 13 Int.64
>  init_chars  704 "    typeid   "
>  init_int  720 2 Int.64
>  init_var  728 v.1 0
>  init_int  736 13 Int.64
>  init_chars  744 "RunMainBody: "
>  init_int  760 2 Int.64
>  init_var  768 v.1 0
>  init_int  776 19 Int.64
>  init_chars  784 "RunMainBody: exec: "
>  init_int  808 2 Int.64
>  init_var  816 v.1 0
>  init_int  824 17 Int.64
>  init_chars  832 "ExpandInitStack: "
>
> MIPS64_OPENBSD:
>
>  init_chars  664 "    typecode "
>  init_int  680 8796093022208 Int.64
>  init_var  688 v.1 0
>  init_int  696 13 Int.64
>  init_chars  704 "    typeid   "
>  init_int  720 8796093022208 Int.64
>  init_var  728 v.1 0
>  init_int  736 13 Int.64
>  init_chars  744 "RunMainBody: "
>  init_int  760 8796093022208 Int.64
>  init_var  768 v.1 0
>  init_int  776 19 Int.64
>  init_chars  784 "RunMainBody: exec: "
>  init_int  808 8796093022208 Int.64
>  init_var  816 v.1 0
>
> This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where  
> little endian has 2.
>
> I don't know what these numbers represent.
> I should look at other systems -- preferably a working 64 bit big  
> endian system, if there are any.
>
> Could be flags/type with text literals?
>
> (Btw, I always leave the TextLiteral.i3 change on my system,  
> lowering the maximum literal length to around 4gig even on 64 bit  
> targets, so 32 bit can bootstrap 64 bit...need to fix the compiler  
> here to do target arithmetic instead of host arithmetic...)
>
> Thanks,
>  - Jay
>
>
> From: hosking at cs.purdue.edu
> To: jay.krell at cornell.edu
> Date: Sat, 8 Nov 2008 22:55:54 +0000
> CC: m3devel at elegosoft.com
> Subject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
>
> Have you checked to make sure that the pre-gcc IR makes sense?  Does  
> it see these values as 32-bit or 64-bit?  The reason I ask is that  
> there may be something wrong in your m3middle target specs that  
> cause bad IR.  Once we know the IR looks right then we can try and  
> fix the problem.
>
> Antony Hosking | Associate Professor | Computer Science | Purdue  
> University
> 305 N. University Street | West Lafayette | IN 47907 | USA
> Office +1 765 494 6001 | Mobile +1 765 427 5484
>
>
>
> On 8 Nov 2008, at 15:45, Jay wrote:
>
>
> truncated as usual...along with losing a lot of whitespace..
> Let's see...
>
>
> Tony, how about switching these #if's?
> I explain some of why below.
>
> static void
> m3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t,  
> m3_type dst_T)
> {
>  tree val;
> #if 0
>  if (o != 0 || TREE_TYPE (v) != dst_t) {
>    v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),
>     bitsize_int (o));
>  }
> #else
>  /* failsafe, but inefficient */
>  if (o != 0 || TREE_TYPE (v) != dst_t) {
>    v = m3_build1 (ADDR_EXPR, t_addr, v);
>    v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));
>    v = m3_build1 (INDIRECT_REF, dst_t,
>     m3_cast (build_pointer_type (dst_t), v));
>  }
> #endif
>  TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing  
> problems */
>  val = m3_cast (src_t, EXPR_REF (-1));
>  if (src_T != dst_T) {
>    val = m3_build1 (CONVERT_EXPR, dst_t, val);
>  }
>  add_stmt (build2 (MODIFY_EXPR, dst_t, v, val));
>  EXPR_POP ();
> }
> m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t,  
> m3_type dst_T)
> {
> #if 0
>  if (o != 0 || TREE_TYPE (v) != src_t) {
>    v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),
>     bitsize_int (o));
>  }
> #else
>  /* failsafe, but inefficient */
>  if (o != 0 || TREE_TYPE (v) != src_t) {
>    v = m3_build1 (ADDR_EXPR, t_addr, v);
>    v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));
>    v = m3_build1 (INDIRECT_REF, src_t,
>     m3_cast (build_pointer_type (src_t), v));
>  }
> #endif
>  TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing  
> problems */
>  if (src_T != dst_T) {
>    v = m3_build1 (CONVERT_EXPR, dst_t, v);
>  }
>  EXPR_PUSH (v);
> }
>
> I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).
> In an early version it seems that many loads/stores of global  
> variables is wrong.
> The "first" code affected is RTLinker_ExpandModuleTable.
>
> PROCEDURE ExpandModuleTable () =
>  CONST InitialTableSize = 500;
>  VAR new_mods: ADDRESS;  n_bytes: INTEGER;
>  BEGIN
>    IF (modules = NIL) THEN
>      (* first time... *)
>      max_modules := InitialTableSize;
>      modules := Cstdlib.malloc (InitialTableSize * BYTESIZE  
> (RT0.ModulePtr));
>      IF (modules = NIL) THEN Cstdlib.abort (); END;
>    ELSE
>      n_bytes := max_modules * BYTESIZE (RT0.ModulePtr);
>      new_mods := Cstdlib.malloc (n_bytes + n_bytes);
>      IF (new_mods = NIL) THEN Cstdlib.abort (); END;
>      EVAL Cstring.memcpy (new_mods, modules, n_bytes);
>      Cstdlib.free (modules);
>      modules := new_mods;
>      INC (max_modules, max_modules);
>    END;
>  END ExpandModuleTable;
>
> It should read back modules as NIL.
> What happens is it reads back modules as not NIL,
> and then max_modules should also be zero, but is "large", leading to  
> malloc to fail,
> and abort called.
>
> The output of cm3cg -y is reasonable.
> It indicates offset loads.
>
> What is incorrect is that in many loads and stores, but not all, the
> offsets are lost.
>
> (923) begin_procedure
>  procedure RTLinker__ExpandModuleTable
> RTLinker__ExpandModuleTable(924) set_source_line
>  source line  208
> (925) load_nil
> (926) store
>  store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb
> (927) set_source_line
>  source line  207
> (928) set_source_line
>  source line  210
> (929) load_nil
> (930) load
>  m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xb
>
> Here is the incorrect code:
>
> .globl RTLinker__ExpandModuleTable
> .stabd 46,0,0
> .stabn 68,0,206,.LM108
> .LM108:
> .LFBB8:
> .set nomips16
> .ent RTLinker__ExpandModuleTable
> RTLinker__ExpandModuleTable:
> .frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0
> .mask 0xd0000000,-8
> .fmask 0x00000000,0
> daddiu $sp,$sp,-64
> sd $31,56($sp)
> sd $fp,48($sp)
> sd $28,40($sp)
> move $fp,$sp
> lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
> daddu $28,$28,$25
> daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
> .LBB9:
> .stabn 68,0,208,.LM109
> .LM109:
> sd $0,16($fp)
> .stabn 68,0,210,.LM110
> .LM110:
> dla $2,MM_RTLinker
> ld $2,0($2)      << WRONG
> bne $2,$0,.L61
> .stabn 68,0,212,.LM111
> .LM111:
> dla $3,MM_RTLinker
> ld $2,128($3)
> andi $2,$2,0x0
> ori $2,$2,0x1f4
> sd $2,128($3)
> .stabn 68,0,213,.LM112
> .LM112:
> li $4,4000   # 0xfa0
> jal malloc
> sd $2,0($fp)
> ld $4,0($fp)
> dla $3,MM_RTLinker
> ld $2,136($3)
> andi $2,$2,0x0
> or $2,$4,$2
> sd $2,136($3)     << OK, offset seems low, but it agrees with  
> elsewhere
> .stabn 68,0,214,.LM113
> .LM113:
> dla $2,MM_RTLinker
> ld $2,0($2)       << WRONG
> bne $2,$0,.L65
> jal abort
> b .L65
> .L61:
> .stabn 68,0,216,.LM114
> .LM114:
> dla $2,MM_RTLinker
> ld $2,0($2)        << WRONG
> dsll $2,$2,3
> sd $2,8($fp)
> .stabn 68,0,217,.LM115
> .LM115:
> ld $3,8($fp)
> ld $2,8($fp)
> daddu $2,$3,$2
> move $4,$2
> jal malloc
> sd $2,0($fp)
> ld $2,0($fp)
> sd $2,16($fp)
> .stabn 68,0,218,.LM116
> .LM116:
> ld $2,16($fp)
> bne $2,$0,.L64
> jal abort
> .L64:
> .stabn 68,0,219,.LM117
> .LM117:
> ld $4,16($fp)
> dla $2,MM_RTLinker
> ld $2,0($2)          << WRONG
> ld $3,8($fp)
> move $5,$2
> move $6,$3
> jal memcpy
> sd $2,0($fp)
> .stabn 68,0,220,.LM118
> .LM118:
> dla $2,MM_RTLinker
> ld $2,0($2)           << WRONG
> move $4,$2
> jal free
> .stabn 68,0,221,.LM119
> .LM119:
> ld $4,16($fp)
> dla $3,MM_RTLinker
> ld $2,136($3)         << OK
> andi $2,$2,0x0
> or $2,$4,$2
> sd $2,136($3)         << OK
> .stabn 68,0,222,.LM120
> .LM120:
> dla $2,MM_RTLinker
> ld $3,0($2)           << WRONG
> dla $2,MM_RTLinker
> ld $2,0($2)           << WRONG
> daddu $4,$3,$2
> dla $3,MM_RTLinker
> ld $2,128($3)         << OK
> andi $2,$2,0x0
> or $2,$2,$4
> sd $2,128($3)         << OK
> .L65:
> .LBE9:
> .stabn 68,0,224,.LM121
> .LM121:
> move $sp,$fp
> ld $31,56($sp)
> ld $fp,48($sp)
> ld $28,40($sp)
> daddiu $sp,$sp,64
> j $31
> .end RTLinker__ExpandModuleTable
>
> Here is corrected code:
>
> RTLinker__ExpandModuleTable:
> .frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0
> .mask 0xd0000000,-8
> .fmask 0x00000000,0
> daddiu $sp,$sp,-64
> sd $31,56($sp)
> sd $fp,48($sp)
> sd $28,40($sp)
> move $fp,$sp
> lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
> daddu $28,$28,$25
> daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
> .LBB9:
> .stabn 68,0,208,.LM109
> .LM109:
> sd $0,16($fp)
> .stabn 68,0,210,.LM110
> .LM110:
> dla $2,MM_RTLinker
> daddiu $2,$2,136     << CORRECT (offset seems low, but at least it  
> isn't zero)
> ld $2,0($2)
> bne $2,$0,.L61
> .stabn 68,0,212,.LM111
> .LM111:
> dla $2,MM_RTLinker
> daddiu $3,$2,128     << CORRECT
> li $2,500   # 0x1f4
> sd $2,0($3)
> .stabn 68,0,213,.LM112
> .LM112:
> li $4,4000   # 0xfa0
> jal malloc
> sd $2,0($fp)
> dla $2,MM_RTLinker
> daddiu $3,$2,136      << CORRECT
> ld $2,0($fp)
> sd $2,0($3)
> .stabn 68,0,214,.LM113
> .LM113:
> dla $2,MM_RTLinker
> daddiu $2,$2,136      << CORRECT
> ld $2,0($2)
> bne $2,$0,.L65
> jal abort
> b .L65
> .L61:
>
> I still dump core with this change, but I get much further.
> I have not tested this change at all otherwise -- i.e. on working  
> platforms.
>
> I have not debugged through the gcc/m3cg code to see why
> the offsets are only sometimes lost, or why other platforms
> have no problem. Is MIPS's "global pointer" unusual these days?
> I know SPARC has similar, and all the non-x86, non-AMD64 NT platforms
> had similar, but that's essentially nothing now (I'm not counting  
> IA64).
> I know Linux has PLT (procedure linkage table) and GOT (global  
> offset table).
> Same thing?
> I haven't looked much at other MIPS ports yet.
>
> http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8
>
> is what made it look about like it does -- in particularing using  
> BIT_FIELD_REF.
>
> 1.7:
>
> static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */
>     tree v;
>     int o;
>     tree src_t, dest_t;
>     m3_type src_T, dest_T;
> {
>  if (o == 0 && TREE_TYPE (v) == src_t) {
>    EXPR_PUSH (v);
>  } else {
>    tree adr = m3_build1 (ADDR_EXPR, t_addr, v);
>    if (o != 0) {
>      adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o /  
> BITS_PER_UNIT));
>    }
>    EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t,
>                          m3_cast (build_pointer_type (src_t), adr)));
>  }
> #if 1
>  if (src_T != dest_T) {
>    EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1));
>  }
>  if (debug_vars) {
>    const char *name = "noname";
>    if (v != 0 && DECL_NAME(v) != 0) {
>      name = IDENTIFIER_POINTER(DECL_NAME(v));
>    }
>    fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n",  
> name,
>            o, src_T, dest_T);
>  }
> #else
>  if (src_T != dest_T) {
>    if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) {
>      EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1));
>    } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) {
>      EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1));
>    } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) {
>      /* okay */
>    } else {
>      fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n",
>                  src_T, dest_T);
>    }
>  }
> #endif
> }
> static void
> m3_store (v, o, src_t, dest_t)  /* GCC32OK */
>     tree v;
>     int o;
>     tree src_t, dest_t;
> {
>  tree lhs, rhs;
>  if (TREE_TYPE (EXPR_REF (-1)) == src_t) {
>    rhs = EXPR_REF (-1);
>  } else {
>    rhs = m3_cast (src_t, EXPR_REF (-1));
>  }
>  if (o == 0 && TREE_TYPE (v) == dest_t) {
>    lhs = v;
>  } else {
>    tree f = make_node (FIELD_DECL);
>    TREE_TYPE (f) = dest_t;
>    DECL_ALIGN (f) = TYPE_ALIGN (dest_t);
>    DECL_SIZE (f) = TYPE_SIZE (dest_t);
>    DECL_MODE (f) = TYPE_MODE (dest_t);
>    DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT);
>    DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT);
>    DECL_FIELD_CONTEXT (f) = TREE_TYPE (v);
>    lhs = m3_build2 (COMPONENT_REF, dest_t, v, f);
>  }
> #if 1
>    expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0,  
> 0);
> #else
>  if (dest_t ==  NULL) {
>    expand_assignment (lhs, rhs, 0, 0);
>  } else {
>    expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0,  
> 0);
>  }
> #endif
>  EXPR_POP ();
> }
>
> 1.8:
> static void
> m3_load (tree v, int o,
>  tree src_t, m3_type src_T,
>  tree dst_t, m3_type dst_T) /* GCC32OK */
> {
>  if (o == 0 && TREE_TYPE (v) == src_t) {
>    EXPR_PUSH (v);
>  } else {
>    EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),
>     bitsize_int (o)));
>  }
>  if (src_T != dst_T) {
>    EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1));
>  }
>  if (option_vars_trace) {
>    const char *name = "noname";
>    if (v != 0 && DECL_NAME(v) != 0) {
>      name = IDENTIFIER_POINTER(DECL_NAME(v));
>    }
>    fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n",  
> name,
>     o, src_T, dst_T);
>  }
> }
> static void
> m3_store (tree v, int o, tree src_t, tree dst_t)  /* GCC32OK */
> {
>  tree lhs, rhs;
>  if (TREE_TYPE (EXPR_REF (-1)) == src_t) {
>    rhs = EXPR_REF (-1);
>  } else {
>    rhs = m3_cast (src_t, EXPR_REF (-1));
>  }
>  if (o == 0 && TREE_TYPE (v) == dst_t) {
>    lhs = v;
>  } else {
>    lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),
>       bitsize_int (o));
>  }
>  if (src_t != dst_t) {
>    rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs);
>  }
>  expand_assignment (lhs, rhs, 0, 0);
>  EXPR_POP ();
> }
>
> - Jay
>
> <AMD64_LINUX_RTLinker.mc><MIPS64_OPENBSD_RTLinker.mc>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081109/b8730591/attachment.html>

From jay.krell at cornell.edu  Sun Nov  9 12:10:54 2008
From: jay.krell at cornell.edu (Jay)
Date: Sun, 9 Nov 2008 11:10:54 +0000
Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
In-Reply-To: <B9A95C1E-B873-40A9-A045-CD38E32CA5D0@cs.purdue.edu>
References: <COL101-W3DB0BA622C4D31A82B602E6180@phx.gbl>
	<C47F6B5A-F154-4485-B839-96185B59E3B6@cs.purdue.edu>
	<COL101-W2CFF8666D1486BD514159E61B0@phx.gbl> 
	<B9A95C1E-B873-40A9-A045-CD38E32CA5D0@cs.purdue.edu>
Message-ID: <BAY123-W4494232D2A947D3041E17AE61B0@phx.gbl>


I agree. I am suspicious too. I will investigate more. Thanks.
 
 - Jay



From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 9 Nov 2008 10:01:58 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
Aren't the integer values supposed to be written endian-independently in the IR?  I am suspicious about these differences.




On 9 Nov 2008, at 01:25, Jay wrote:

Thanks for the tip/direction. Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the same" but for two systematic diffs:  - AMD64_LINUX has a bunch of added calls to check_nil.   This is a "bug" in it, inoptimal but ok code, because "first readable address" is 0. It should probably be 4k. I should write a program that iterates starting at 0 to find the value.  (or better yet, cm3 should have a -configure option that does this? -- for configuration values for which there is a safe default; in general, imho, the targets should be auto configured via C, assuming there is a native development environment available).  - some integers are different (again, systematically)  AMD64_LINUX: init_int  656 13 Int.64 init_chars  664 "    typecode " init_int  680 2 Int.64 init_var  688 v.1 0 init_int  696 13 Int.64 init_chars  704 "    typeid   " init_int  720 2 Int.64 init_var  728 v.1 0 init_int  736 13 Int.64 init_chars  744 "RunMainBody: " init_int  760 2 Int.64 init_var  768 v.1 0 init_int  776 19 Int.64 init_chars  784 "RunMainBody: exec: " init_int  808 2 Int.64 init_var  816 v.1 0 init_int  824 17 Int.64 init_chars  832 "ExpandInitStack: "MIPS64_OPENBSD:  init_chars  664 "    typecode " init_int  680 8796093022208 Int.64 init_var  688 v.1 0 init_int  696 13 Int.64 init_chars  704 "    typeid   " init_int  720 8796093022208 Int.64 init_var  728 v.1 0 init_int  736 13 Int.64 init_chars  744 "RunMainBody: " init_int  760 8796093022208 Int.64 init_var  768 v.1 0 init_int  776 19 Int.64 init_chars  784 "RunMainBody: exec: " init_int  808 8796093022208 Int.64 init_var  816 v.1 0This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where little endian has 2. I don't know what these numbers represent.I should look at other systems -- preferably a working 64 bit big endian system, if there are any. Could be flags/type with text literals? (Btw, I always leave the TextLiteral.i3 change on my system, lowering the maximum literal length to around 4gig even on 64 bit targets, so 32 bit can bootstrap 64 bit...need to fix the compiler here to do target arithmetic instead of host arithmetic...) Thanks, - Jay

From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sat, 8 Nov 2008 22:55:54 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Have you checked to make sure that the pre-gcc IR makes sense?  Does it see these values as 32-bit or 64-bit?  The reason I ask is that there may be something wrong in your m3middle target specs that cause bad IR.  Once we know the IR looks right then we can try and fix the problem.



Antony Hosking | Associate Professor | Computer Science | Purdue University
305 N. University Street | West Lafayette | IN 47907 | USA
Office +1 765 494 6001 | Mobile +1 765 427 5484


On 8 Nov 2008, at 15:45, Jay wrote:

truncated as usual...along with losing a lot of whitespace..Let's see...Tony, how about switching these #if's?I explain some of why below.static voidm3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){ tree val;#if 0 if (o != 0 || TREE_TYPE (v) != dst_t) {   v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),    bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) {   v = m3_build1 (ADDR_EXPR, t_addr, v);   v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));   v = m3_build1 (INDIRECT_REF, dst_t,    m3_cast (build_pointer_type (dst_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) {   val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP ();}m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){#if 0 if (o != 0 || TREE_TYPE (v) != src_t) {   v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),    bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) {   v = m3_build1 (ADDR_EXPR, t_addr, v);   v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));   v = m3_build1 (INDIRECT_REF, src_t,    m3_cast (build_pointer_type (src_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) {   v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v);}I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).In an early version it seems that many loads/stores of global variables is wrong.The "first" code affected is RTLinker_ExpandModuleTable.PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS;  n_bytes: INTEGER; BEGIN   IF (modules = NIL) THEN     (* first time... *)     max_modules := InitialTableSize;     modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr));     IF (modules = NIL) THEN Cstdlib.abort (); END;   ELSE     n_bytes := max_modules * BYTESIZE (RT0.ModulePtr);     new_mods := Cstdlib.malloc (n_bytes + n_bytes);     IF (new_mods = NIL) THEN Cstdlib.abort (); END;     EVAL Cstring.memcpy (new_mods, modules, n_bytes);     Cstdlib.free (modules);     modules := new_mods;     INC (max_modules, max_modules);   END; END ExpandModuleTable;It should read back modules as NIL.What happens is it reads back modules as not NIL,and then max_modules should also be zero, but is "large", leading to malloc to fail,and abort called.The output of cm3cg -y is reasonable.It indicates offset loads.What is incorrect is that in many loads and stores, but not all, theoffsets are lost.(923) begin_procedure procedure RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable(924) set_source_line source line  208(925) load_nil(926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb(927) set_source_line source line  207(928) set_source_line source line  210(929) load_nil(930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xbHere is the incorrect code:.globl RTLinker__ExpandModuleTable.stabd 46,0,0.stabn 68,0,206,.LM108.LM108:.LFBB8:.set nomips16.ent RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable:.frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerld $2,0($2)      << WRONGbne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $3,MM_RTLinkerld $2,128($3)andi $2,$2,0x0ori $2,$2,0x1f4sd $2,128($3).stabn 68,0,213,.LM112.LM112:li $4,4000   # 0xfa0jal mallocsd $2,0($fp)ld $4,0($fp)dla $3,MM_RTLinkerld $2,136($3)andi $2,$2,0x0or $2,$4,$2sd $2,136($3)     << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerld $2,0($2)       << WRONG bne $2,$0,.L65jal abortb .L65.L61:.stabn 68,0,216,.LM114.LM114:dla $2,MM_RTLinkerld $2,0($2)        << WRONG dsll $2,$2,3sd $2,8($fp).stabn 68,0,217,.LM115.LM115:ld $3,8($fp)ld $2,8($fp)daddu $2,$3,$2move $4,$2jal mallocsd $2,0($fp)ld $2,0($fp)sd $2,16($fp).stabn 68,0,218,.LM116.LM116:ld $2,16($fp)bne $2,$0,.L64jal abort.L64:.stabn 68,0,219,.LM117.LM117:ld $4,16($fp)dla $2,MM_RTLinkerld $2,0($2)          << WRONGld $3,8($fp)move $5,$2move $6,$3jal memcpysd $2,0($fp).stabn 68,0,220,.LM118.LM118:dla $2,MM_RTLinkerld $2,0($2)           << WRONGmove $4,$2jal free.stabn 68,0,221,.LM119.LM119:ld $4,16($fp)dla $3,MM_RTLinkerld $2,136($3)         << OKandi $2,$2,0x0or $2,$4,$2sd $2,136($3)         << OK .stabn 68,0,222,.LM120.LM120:dla $2,MM_RTLinkerld $3,0($2)           << WRONGdla $2,MM_RTLinkerld $2,0($2)           << WRONGdaddu $4,$3,$2dla $3,MM_RTLinkerld $2,128($3)         << OKandi $2,$2,0x0or $2,$2,$4sd $2,128($3)         << OK.L65:.LBE9:.stabn 68,0,224,.LM121.LM121:move $sp,$fpld $31,56($sp)ld $fp,48($sp)ld $28,40($sp)daddiu $sp,$sp,64j $31.end RTLinker__ExpandModuleTableHere is corrected code:RTLinker__ExpandModuleTable:.frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerdaddiu $2,$2,136     << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2)bne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $2,MM_RTLinkerdaddiu $3,$2,128     << CORRECT li $2,500   # 0x1f4sd $2,0($3).stabn 68,0,213,.LM112.LM112:li $4,4000   # 0xfa0jal mallocsd $2,0($fp)dla $2,MM_RTLinkerdaddiu $3,$2,136      << CORRECT ld $2,0($fp)sd $2,0($3).stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerdaddiu $2,$2,136      << CORRECT ld $2,0($2)bne $2,$0,.L65jal abortb .L65.L61:I still dump core with this change, but I get much further.I have not tested this change at all otherwise -- i.e. on working platforms.I have not debugged through the gcc/m3cg code to see whythe offsets are only sometimes lost, or why other platformshave no problem. Is MIPS's "global pointer" unusual these days?I know SPARC has similar, and all the non-x86, non-AMD64 NT platformshad similar, but that's essentially nothing now (I'm not counting IA64).I know Linux has PLT (procedure linkage table) and GOT (global offset table).Same thing?I haven't looked much at other MIPS ports yet.http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8is what made it look about like it does -- in particularing using BIT_FIELD_REF.1.7:static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */    tree v;    int o;    tree src_t, dest_t;    m3_type src_T, dest_T;{ if (o == 0 && TREE_TYPE (v) == src_t) {   EXPR_PUSH (v); } else {   tree adr = m3_build1 (ADDR_EXPR, t_addr, v);   if (o != 0) {     adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT));   }   EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t,                         m3_cast (build_pointer_type (src_t), adr))); }#if 1 if (src_T != dest_T) {   EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) {   const char *name = "noname";   if (v != 0 && DECL_NAME(v) != 0) {     name = IDENTIFIER_POINTER(DECL_NAME(v));   }   fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n", name,           o, src_T, dest_T); }#else if (src_T != dest_T) {   if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) {     EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1));   } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) {     EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1));   } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) {     /* okay */   } else {     fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n",                 src_T, dest_T);   } }#endif}static voidm3_store (v, o, src_t, dest_t)  /* GCC32OK */    tree v;    int o;    tree src_t, dest_t;{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) {   rhs = EXPR_REF (-1); } else {   rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) {   lhs = v; } else {   tree f = make_node (FIELD_DECL);   TREE_TYPE (f) = dest_t;   DECL_ALIGN (f) = TYPE_ALIGN (dest_t);   DECL_SIZE (f) = TYPE_SIZE (dest_t);   DECL_MODE (f) = TYPE_MODE (dest_t);   DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT);   DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT);   DECL_FIELD_CONTEXT (f) = TREE_TYPE (v);   lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); }#if 1   expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);#else if (dest_t ==  NULL) {   expand_assignment (lhs, rhs, 0, 0); } else {   expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); }#endif EXPR_POP ();}1.8:static voidm3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */{ if (o == 0 && TREE_TYPE (v) == src_t) {   EXPR_PUSH (v); } else {   EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),    bitsize_int (o))); } if (src_T != dst_T) {   EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) {   const char *name = "noname";   if (v != 0 && DECL_NAME(v) != 0) {     name = IDENTIFIER_POINTER(DECL_NAME(v));   }   fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n", name,    o, src_T, dst_T); }}static voidm3_store (tree v, int o, tree src_t, tree dst_t)  /* GCC32OK */{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) {   rhs = EXPR_REF (-1); } else {   rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) {   lhs = v; } else {   lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),      bitsize_int (o)); } if (src_t != dst_t) {   rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP ();}- Jay<AMD64_LINUX_RTLinker.mc><MIPS64_OPENBSD_RTLinker.mc>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081109/f520ce42/attachment.html>

From jay.krell at cornell.edu  Sun Nov  9 15:09:10 2008
From: jay.krell at cornell.edu (Jay)
Date: Sun, 9 Nov 2008 14:09:10 +0000
Subject: [M3devel] AMD64_LINUX gc/stubgen
Message-ID: <COL101-W14DAC6D23849CBD1BE010AE61B0@phx.gbl>


Tony, any tips on this one? :)
 
jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 -sno RemoteView.T -T.M3IMPTAB @M3nogc
 
always succeeds.
 
jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 -sno RemoteView.T   -T.M3IMPTAB
 
usually fails:
****** runtime error:***    NEW() was unable to allocate more memory.***    file "../src/runtime/common/RTAllocator.m3", line 285***Formatter.m3:
 
CONST ChunkSize = 128; (* Must be a power of 2 *)
PROCEDURE New(wr: Wr.T;  width: CARDINAL:= 75): T RAISES {} =  BEGIN    WITH t = NEW(T) DO      t.wr := wr;      t.width := width;      t.indent := 0;        t.numChars := 0;        (* t.stream := RefStream{NIL, ..}; *)        t.buffer := NEW (ExprBuf, 4 * ChunkSize);  << this is the line that fails
 
Lowering chunk size to 64 seemed to make it succeed more often, but not always.
I think I'll go back to MIPS64_OPENBSD..probably easier to track that problem down.
 
 - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081109/00d5c623/attachment.html>

From hosking at cs.purdue.edu  Sun Nov  9 19:02:41 2008
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Sun, 9 Nov 2008 18:02:41 +0000
Subject: [M3devel] AMD64_LINUX gc/stubgen
In-Reply-To: <COL101-W14DAC6D23849CBD1BE010AE61B0@phx.gbl>
References: <COL101-W14DAC6D23849CBD1BE010AE61B0@phx.gbl>
Message-ID: <97806969-D1C7-4D77-8B54-466088B99937@cs.purdue.edu>

Clearly a problem with GC.  I'm guessing you have something wrong  
(pointer alignment?) in the configuration.  I suspect if you run with  
@M3paranoidgc you'll trip over the problem.

On 9 Nov 2008, at 14:09, Jay wrote:

> Tony, any tips on this one? :)
>
> jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 - 
> sno RemoteView.T -T.M3IMPTAB @M3nogc
>
> always succeeds.
>
> jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 - 
> sno RemoteView.T   -T.M3IMPTAB
>
> usually fails:
>
> ***
> *** runtime error:
> ***    NEW() was unable to allocate more memory.
> ***    file "../src/runtime/common/RTAllocator.m3", line 285
> ***
>
> Formatter.m3:
>
> CONST ChunkSize = 128; (* Must be a power of 2 *)
>
> PROCEDURE New(wr: Wr.T;  width: CARDINAL:= 75): T RAISES {} =
>   BEGIN
>     WITH t = NEW(T) DO
>       t.wr := wr;
>       t.width := width;
>       t.indent := 0;
>
>       t.numChars := 0;
>
>       (* t.stream := RefStream{NIL, ..}; *)
>
>       t.buffer := NEW (ExprBuf, 4 * ChunkSize);  << this is the line  
> that fails
>
> Lowering chunk size to 64 seemed to make it succeed more often, but  
> not always.
> I think I'll go back to MIPS64_OPENBSD..probably easier to track  
> that problem down.
>
>  - Jay

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081109/623733d0/attachment.html>

From jay.krell at cornell.edu  Mon Nov 10 09:28:18 2008
From: jay.krell at cornell.edu (Jay)
Date: Mon, 10 Nov 2008 08:28:18 +0000
Subject: [M3devel] AMD64_LINUX gc/stubgen
In-Reply-To: <97806969-D1C7-4D77-8B54-466088B99937@cs.purdue.edu>
References: <COL101-W14DAC6D23849CBD1BE010AE61B0@phx.gbl>
	<97806969-D1C7-4D77-8B54-466088B99937@cs.purdue.edu>
Message-ID: <COL101-W13EBD249838CF77D46F8D2E61A0@phx.gbl>


@M3paranoidgc always succeeds. I see, reading the code, it only does reads. Nevertheless.
 
Simply programs that just allocate memory forever also succeed.
Such as this:
 
jay at amd64a:/dev2/cm3/t$ cat src/t.m3MODULE t EXPORTS Main;
TYPE T1 = RECORD c : ARRAY [0..10000000] OF INTEGER; END;TYPE T2 = RECORD c : ARRAY [0..4095] OF INTEGER; END;
 
VAR a: REFANY;
 
PROCEDURE F1()=VAR b: REFANY;
BEGIN  WHILE TRUE DO    b := NEW(REF T2);    a := NEW(REF T1);  END;END F1;
 
BEGIN  F1();END t.
 
Darn.
 
 - Jay



From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 9 Nov 2008 18:02:41 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] AMD64_LINUX gc/stubgen
Clearly a problem with GC.  I'm guessing you have something wrong (pointer alignment?) in the configuration.  I suspect if you run with @M3paranoidgc you'll trip over the problem.




On 9 Nov 2008, at 14:09, Jay wrote:

Tony, any tips on this one? :) jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 -sno RemoteView.T -T.M3IMPTAB @M3nogc always succeeds. jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 -sno RemoteView.T   -T.M3IMPTAB usually fails:****** runtime error:***    NEW() was unable to allocate more memory.***    file "../src/runtime/common/RTAllocator.m3", line 285***Formatter.m3: CONST ChunkSize = 128; (* Must be a power of 2 *)PROCEDURE New(wr: Wr.T;  width: CARDINAL:= 75): T RAISES {} =  BEGIN    WITH t = NEW(T) DO      t.wr := wr;      t.width := width;      t.indent := 0;        t.numChars := 0;        (* t.stream := RefStream{NIL, ..}; *)        t.buffer := NEW (ExprBuf, 4 * ChunkSize);  << this is the line that fails Lowering chunk size to 64 seemed to make it succeed more often, but not always.I think I'll go back to MIPS64_OPENBSD..probably easier to track that problem down.  - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081110/da2e695b/attachment.html>

From hosking at cs.purdue.edu  Mon Nov 10 12:13:05 2008
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Mon, 10 Nov 2008 11:13:05 +0000
Subject: [M3devel] AMD64_LINUX gc/stubgen
In-Reply-To: <COL101-W13EBD249838CF77D46F8D2E61A0@phx.gbl>
References: <COL101-W14DAC6D23849CBD1BE010AE61B0@phx.gbl>
	<97806969-D1C7-4D77-8B54-466088B99937@cs.purdue.edu>
	<COL101-W13EBD249838CF77D46F8D2E61A0@phx.gbl>
Message-ID: <831F958E-6D14-4DDD-AEF1-7BF3AAA3BC1B@cs.purdue.edu>

Hmm.  Can you try @M3paranoidgc @M3noincremental @M3nogenerational?

On 10 Nov 2008, at 08:28, Jay wrote:

> @M3paranoidgc always succeeds. I see, reading the code, it only does  
> reads. Nevertheless.
>
> Simply programs that just allocate memory forever also succeed.
> Such as this:
>
> jay at amd64a:/dev2/cm3/t$ cat src/t.m3
> MODULE t EXPORTS Main;
> TYPE T1 = RECORD c : ARRAY [0..10000000] OF INTEGER; END;
> TYPE T2 = RECORD c : ARRAY [0..4095] OF INTEGER; END;
>
> VAR a: REFANY;
>
> PROCEDURE F1()=
> VAR b: REFANY;
> BEGIN
>   WHILE TRUE DO
>     b := NEW(REF T2);
>     a := NEW(REF T1);
>   END;
> END F1;
>
> BEGIN
>   F1();
> END t.
>
> Darn.
>
>  - Jay
>
>
> From: hosking at cs.purdue.edu
> To: jay.krell at cornell.edu
> Date: Sun, 9 Nov 2008 18:02:41 +0000
> CC: m3devel at elegosoft.com
> Subject: Re: [M3devel] AMD64_LINUX gc/stubgen
>
>
> Clearly a problem with GC.  I'm guessing you have something wrong  
> (pointer alignment?) in the configuration.  I suspect if you run  
> with @M3paranoidgc you'll trip over the problem.
>
> On 9 Nov 2008, at 14:09, Jay wrote:
>
> Tony, any tips on this one? :)
>
> jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 - 
> sno RemoteView.T -T.M3IMPTAB @M3nogc
>
> always succeeds.
>
> jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 - 
> sno RemoteView.T   -T.M3IMPTAB
>
> usually fails:
>
> ***
> *** runtime error:
> ***    NEW() was unable to allocate more memory.
> ***    file "../src/runtime/common/RTAllocator.m3", line 285
> ***
>
> Formatter.m3:
>
> CONST ChunkSize = 128; (* Must be a power of 2 *)
>
> PROCEDURE New(wr: Wr.T;  width: CARDINAL:= 75): T RAISES {} =
>   BEGIN
>     WITH t = NEW(T) DO
>       t.wr := wr;
>       t.width := width;
>       t.indent := 0;
>
>       t.numChars := 0;
>
>       (* t.stream := RefStream{NIL, ..}; *)
>
>       t.buffer := NEW (ExprBuf, 4 * ChunkSize);  << this is the line  
> that fails
>
> Lowering chunk size to 64 seemed to make it succeed more often, but  
> not always.
> I think I'll go back to MIPS64_OPENBSD..probably easier to track  
> that problem down.
>
>  - Jay
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081110/d1ad8649/attachment.html>

From jay.krell at cornell.edu  Mon Nov 10 20:35:19 2008
From: jay.krell at cornell.edu (Jay)
Date: Mon, 10 Nov 2008 19:35:19 +0000
Subject: [M3devel] AMD64_LINUX gc/stubgen
In-Reply-To: <831F958E-6D14-4DDD-AEF1-7BF3AAA3BC1B@cs.purdue.edu>
References: <COL101-W14DAC6D23849CBD1BE010AE61B0@phx.gbl>
	<97806969-D1C7-4D77-8B54-466088B99937@cs.purdue.edu>
	<COL101-W13EBD249838CF77D46F8D2E61A0@phx.gbl> 
	<831F958E-6D14-4DDD-AEF1-7BF3AAA3BC1B@cs.purdue.edu>
Message-ID: <COL101-W35EAC45CFEB850BBEEFDEDE61A0@phx.gbl>


It /seems/ like any switch or switch combination makes it stop happening.
Admittedly I haven't run a loop just a few runs by hand for each combo.
 
Maybe have it record the allocation quantities and "play them back"?
 
 - Jay


From: hosking at cs.purdue.edu
To: jay.krell at cornell.edu
Date: Mon, 10 Nov 2008 11:13:05 +0000
CC: m3devel at elegosoft.com
Subject: Re: [M3devel] AMD64_LINUX gc/stubgen




Hmm.  Can you try @M3paranoidgc @M3noincremental @M3nogenerational?


On 10 Nov 2008, at 08:28, Jay wrote:


@M3paranoidgc always succeeds. I see, reading the code, it only does reads. Nevertheless.
 
Simply programs that just allocate memory forever also succeed.
Such as this:
 
jay at amd64a:/dev2/cm3/t$ cat src/t.m3
MODULE t EXPORTS Main;
TYPE T1 = RECORD c : ARRAY [0..10000000] OF INTEGER; END;
TYPE T2 = RECORD c : ARRAY [0..4095] OF INTEGER; END;
 
VAR a: REFANY;
 
PROCEDURE F1()=
VAR b: REFANY;
BEGIN
  WHILE TRUE DO
    b := NEW(REF T2);
    a := NEW(REF T1);
  END;
END F1;
 
BEGIN
  F1();
END t.
 
Darn.
 
 - Jay






From: hosking at cs.purdue.edu
To: jay.krell at cornell.edu
Date: Sun, 9 Nov 2008 18:02:41 +0000
CC: m3devel at elegosoft.com
Subject: Re: [M3devel] AMD64_LINUX gc/stubgen



Clearly a problem with GC.  I'm guessing you have something wrong (pointer alignment?) in the configuration.  I suspect if you run with @M3paranoidgc you'll trip over the problem.





On 9 Nov 2008, at 14:09, Jay wrote:


Tony, any tips on this one? :)
 
jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 -sno RemoteView.T -T.M3IMPTAB @M3nogc
 
always succeeds.
 
jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 -sno RemoteView.T   -T.M3IMPTAB
 
usually fails:

***
*** runtime error:
***    NEW() was unable to allocate more memory.
***    file "../src/runtime/common/RTAllocator.m3", line 285
***

Formatter.m3:
 
CONST ChunkSize = 128; (* Must be a power of 2 *)

PROCEDURE New(wr: Wr.T;  width: CARDINAL:= 75): T RAISES {} =
  BEGIN
    WITH t = NEW(T) DO
      t.wr := wr;
      t.width := width;
      t.indent := 0;
  
      t.numChars := 0;
  
      (* t.stream := RefStream{NIL, ..}; *)
  
      t.buffer := NEW (ExprBuf, 4 * ChunkSize);  << this is the line that fails
 
Lowering chunk size to 64 seemed to make it succeed more often, but not always.
I think I'll go back to MIPS64_OPENBSD..probably easier to track that problem down.
 
 - Jay

From rcoleburn at scires.com  Tue Nov 11 00:17:31 2008
From: rcoleburn at scires.com (Randy Coleburn)
Date: Mon, 10 Nov 2008 18:17:31 -0500
Subject: [M3devel] [M3commit] CVS Update: cm3
In-Reply-To: <20081109062016.407EF10D5894@birch.elegosoft.com>
References: <20081109062016.407EF10D5894@birch.elegosoft.com>
Message-ID: <491879AF.1E75.00D7.1@scires.com>

Jay:
 
Does this mean we will now need to put a "cm3cg.common" file into our bin folder on Windows?  
 
Just checking before I do any updates from the repository so I know what needs to change in my target .
 
Regards,
Randy

>>> Jay Krell <jkrell at elego.de> 11/9/2008 1:26 AM >>>
CVSROOT:/usr/cvs
Changes by:jkrell at birch.08/11/09 07:20:15

Modified files:
cm3/m3-sys/cminstall/src/config/: NT386.common cm3.cfg 
cm3/m3-sys/cminstall/src/config-no-install/: AMD64_LINUX 
                                             LINUXLIBC6 
                                             PPC32_OPENBSD 
                                             PPC_DARWIN 
                                             PPC_LINUX SOLgnu 
                                             SOLsun 
                                             SPARC32_LINUX 
                                             SPARC64_LINUX 
                                             SPARC64_OPENBSD 
                                             Unix.common 
Added files:
cm3/m3-sys/cminstall/src/config-no-install/: MIPS64_OPENBSD 
                                             cm3cfg.common 

Log message:
begin combining nt386.common and unix.common (into cm3cg.common)
mips64_openbsd support
being migrating config-no-install/solgnu and solsun to Unix.common
(had this sitting around, in intermediate state)
go back to old formating because that is all the old C written m3build could handle
add probing for host/target specific cm3cg (so I can stop copying it around
as I switch targets)


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081110/03748963/attachment.html>

From jay.krell at cornell.edu  Tue Nov 11 02:06:07 2008
From: jay.krell at cornell.edu (Jay)
Date: Tue, 11 Nov 2008 01:06:07 +0000
Subject: [M3devel] [M3commit] CVS Update: cm3
In-Reply-To: <491879AF.1E75.00D7.1@scires.com>
References: <20081109062016.407EF10D5894@birch.elegosoft.com>
	<491879AF.1E75.00D7.1@scires.com>
Message-ID: <COL101-W586BB50F95A3C239AE2386E6150@phx.gbl>


If you use my config files, yes.
Note that there are two of them, one just includes the other.
Take "the other".
This is just because on my system, I think:
\cm3\cm3.cfg -- probes around 
 => \dev2\cm3.2\m3-sys\cminstall\src\config\NT386 
 => \dev2\cm3.2\m3-sys\cminstall\src\config\NT386.common  
 => \dev2\cm3.2\m3-sys\cminstall\src\config\cm3cfg.common 
 => \dev2\cm3.2\m3-sys\cminstall\src\config-no-install\cm3cfg.common 
 
NT386 is the oddball for me, since for any other targets, for me, it goes:
 
 \cm3\cm3.cfg -- probes around 
  => \dev2\cm3.2\m3-sys\cminstall\src\config-no-install\platform 
  => \dev2\cm3.2\m3-sys\cminstall\src\config-no-install\Unix.common 
  => \dev2\cm3.2\m3-sys\cminstall\src\config-no-install\cm3cfg.common 
 
I'd like to move NT386* to config-no-install, but I avoided doing so so far.
I'd like to replace config with config-no-install, but avoid that too.
 
 - Jay


Date: Mon, 10 Nov 2008 18:17:31 -0500From: rcoleburn at scires.comTo: m3devel at elegosoft.comSubject: Re: [M3devel] [M3commit] CVS Update: cm3
Jay:
 
Does this mean we will now need to put a "cm3cg.common" file into our bin folder on Windows?  
 
Just checking before I do any updates from the repository so I know what needs to change in my target .
 
Regards,
Randy>>> Jay Krell <jkrell at elego.de> 11/9/2008 1:26 AM >>>CVSROOT:/usr/cvsChanges by:jkrell at birch.08/11/09 07:20:15Modified files:cm3/m3-sys/cminstall/src/config/: NT386.common cm3.cfg cm3/m3-sys/cminstall/src/config-no-install/: AMD64_LINUX                                              LINUXLIBC6                                              PPC32_OPENBSD                                              PPC_DARWIN                                              PPC_LINUX SOLgnu                                              SOLsun                                              SPARC32_LINUX                                              SPARC64_LINUX                                              SPARC64_OPENBSD                                              Unix.common Added files:cm3/m3-sys/cminstall/src/config-no-install/: MIPS64_OPENBSD                                              cm3cfg.common Log message:begin combining nt386.common and unix.common (into cm3cg.common)mips64_openbsd supportbeing migrating config-no-install/solgnu and solsun to Unix.common(had this sitting around, in intermediate state)go back to old formating because that is all the old C written m3build could handleadd probing for host/target specific cm3cg (so I can stop copying it aroundas I switch targets)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081111/4d459506/attachment.html>

From rodney.bates at wichita.edu  Tue Nov 11 21:39:09 2008
From: rodney.bates at wichita.edu (Rodney M. Bates)
Date: Tue, 11 Nov 2008 14:39:09 -0600
Subject: [M3devel] Still can't bootstrap cm3
Message-ID: <4919ED6D.6000700@wichita.edu>

We still have serious problems with the cm3 system building.
I have again gotten into a big hampster wheel of several different
failures, and still can't rebuild a working compiler.
I have gone through enough steps that I am not just positive
I have remembered it all right, but it was something like this:

I started with ./do-cm3-front.sh, ./do-cm3-front.sh -ship, and
./install-cm3-compiler.sh upgrade.  That left me with a compiler
that always failed with:

------------------------------------------------------------------------------------------------------------

rodney at yellowstone:~/proj/m3/cm3-new/cm3/m3-sys/m3middle$ cm3 -build
--- building in LINUXLIBC6 ---

ignoring ../src/m3overrides

"/usr/local/cm3/pkg/libm3/src/bundleintf/bundle-ov.tmpl", line 4: quake runtime 
error: undefined variable: ROOT

--procedure--  -line-  -file---
_import_template       4  /usr/local/cm3/pkg/libm3/src/bundleintf/bundle-ov.tmpl
_import_template       4  /usr/local/cm3/pkg/libm3/src/bundleintf/bundle-ov.tmpl
import            274  /usr/local/cm3/pkg/libm3/LINUXLIBC6/.M3EXPORTS
include_dir        11 
/home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3middle/src/m3makefile
                     4 
/home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3middle/LINUXLIBC6/m3make.args

Fatal Error: package build failed

-----------------------------------------------------------------------------------------------------------

Then I tried rebuilding using various do-cm3-* scripts, including after
supplying argument realclean (which, I have learned many times in the
past, is essential to avoid all sorts of problems), with the same results.
Somewhere in here, I started getting the bad version stamp failures too.
Eventually, trying to rebootstrap with ./upgrade.sh, and also manually,
a package at a time, I was consistently getting:

--------------------------------------------------------------------------------------------------------

rodney at yellowstone:~/proj/m3/cm3-new/cm3/m3-sys/m3middle$ cm3 -build
--- building in LINUXLIBC6 ---

ignoring ../src/m3overrides

"/usr/local/cm3/pkg/libm3/LINUXLIBC6/.M3EXPORTS", line 4: quake runtime error: 
unable to open 
"/home/rodney/proj/m3/cm3-new/cm3/m3-libs/m3core/LINUXLIBC6/.M3EXPORTS" for reading

--procedure--  -line-  -file---
import_version      --  <builtin>
import              4  /usr/local/cm3/pkg/libm3/LINUXLIBC6/.M3EXPORTS
include_dir        11 
/home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3middle/src/m3makefile
                     4 
/home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3middle/LINUXLIBC6/m3make.args

Fatal Error: package build failed

-----------------------------------------------------------------------------------------------------------

Perhaps it was the realclean that deleted all the .M3EXPORTS files.
../boot_cm3-core.sh LINUXLIBC6 gave this failure:

-----------------------------------------------------------------------------------------------------------

rodney at yellowstone:~/proj/m3/cm3-new/cm3/scripts$ ./boot-cm3-core.sh
please specify a cross compilation target platform
rodney at yellowstone:~/proj/m3/cm3-new/cm3/scripts$ ./boot-cm3-core.sh LINUXLIBC6
/home/rodney/proj/m3/cm3-new/cm3/scripts/pkgmap.sh -c "cm3 -build -override 
-DROOT='/home/rodney/proj/m3/cm3-new/cm3' -DCM3_VERSION_TEXT='d5.7.0' 
-DCM3_VERSION_NUMBER='050700' -DCM3_LAST_CHANGED='2008-03-16' 
-DM3_BOOTSTRAP=TRUE -keep" m3core libm3 sysutils m3middle m3objfile m3linker 
m3back m3front m3quake cm3 m3scanner m3tools m3cgcat m3cggen m3bundle bitvector 
digraph parseparams realgeometry set slisp sortedtableextras table-list tempfiles
=== package /home/rodney/proj/m3/cm3-new/cm3/m3-libs/m3core ===
  +++ cm3 -build -override -DROOT='/home/rodney/proj/m3/cm3-new/cm3' 
-DCM3_VERSION_TEXT='d5.7.0' -DCM3_VERSION_NUMBER='050700' 
-DCM3_LAST_CHANGED='2008-03-16' -DM3_BOOTSTRAP=TRUE -keep +++
--- building in LINUXLIBC6 ---

new source -> compiling RTHooks.i3
/bin/sh: 
/home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3cc/LINUXLIBC6-LINUXLIBC6/cm3cg: not found
   m3_backend => 127
m3cc (aka cm3cg) failed compiling: RTHooks.ic

---------------------------------------------------------------------------------------------------------

Fortunately, an earlier ./upgrade.sh had built a cm3cg (though it had failed
on all the Modula-3 code).  I got around the above by putting in a symbolic
link from cm3/m3-sys/m3cc/LINUXLIBC6-LINUXLIBC6 to cm3/m3-sys/m3cc/LINUXLIBC6,
so it could find cm3cg.

Then I went through the package-at-a-time order Tony had given in an old post
to the m3devel list.  That had other problems.  Running cm3 in the package
directory failed with the undefined ROOT problem, but do-pkg.sh would do the
build.  But do-pkg.sh doesn't appear to do a ship, so I had to do that in
each package with cm3 -ship.

After building and shipping m3 middle and m3front, package cm3 itself wouldn't
link, wanting libraries for m3back, m3objfile, m3quake, sysutils, m3linker, libm3,
and m3core.  I skipped it and m3gc*, which don't exist now, and went through the
rest of the order, which completed without failures building.  But that gave
me a compiler that fails with a new problem I have never seen before:

---------------------------------------------------------------------------------------------------------

rodney at yellowstone:~/proj/m3/cm3-new/cm3/scripts$ ./install-cm3-compiler.sh upgrade
cp_if: /usr/local/cm3/bin/cm3 and /usr/local/cm3/bin/cm3-d5.7.0 identical
cp_if: /usr/local/cm3/bin/cm3cg and /usr/local/cm3/bin/cm3cg-d5.7.0 identical


***
*** runtime error:
***    A compile-time type is missing.
***    file "../src/Utils.m3"
***

Aborted (core dumped)
cp /home/rodney/proj/m3/cm3-new/cm3/m3-sys/cm3/LINUXLIBC6/cm3 
/usr/local/cm3/bin/cm3-
cp /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3cc/LINUXLIBC6/cm3cg 
/usr/local/cm3/bin/cm3cg-
cp /usr/local/cm3/bin/cm3- /usr/local/cm3/bin/cm3
cp /usr/local/cm3/bin/cm3cg- /usr/local/cm3/bin/cm3cg

------------------------------------------------------------------------------------------------------------

Going back to an older cm3 makes that go away and makes the ROOT problem
recur.  Working around that with cm3 -DROOT="/home/rodney/proj/m3/cm3-new/cm3"
get the compiler to run, but it doesn't do anything, because it thinks
nothing needs to be recompiled.  I could delete stuff, but I fear that would
just start me around another iteration of the same thing.

I don't believe I have ever gotten a bootstrap to work.  I think I have always
had to download a recent binary and go from there.

Any help would be greatly appreciated.

Rodney Bates



From jay.krell at cornell.edu  Wed Nov 12 05:04:44 2008
From: jay.krell at cornell.edu (Jay)
Date: Wed, 12 Nov 2008 04:04:44 +0000
Subject: [M3devel] Still can't bootstrap cm3
In-Reply-To: <4919ED6D.6000700@wichita.edu>
References: <4919ED6D.6000700@wichita.edu>
Message-ID: <COL101-W81AEB201A820491B221953E6140@phx.gbl>


Try starting from a recent snapshot, not necessarily a recent release, and running upgrade.sh or upgrade.py.

 
ROOT should be set in the config file, or as you showed, on the command line.
upgrade.sh/upgrade.py will figure it out and set it on the command line.
I'd kind of like the compiler to set it itself, but that is kind of inappropriate.


It's definitely possible that starting from too old of a release won't work.

 
Avoid any script with "boot" in the name.
Those are I believe for bringing up new targets and esp. cross building, not for
updating mature working systems that can build natively.
The dash in LINUXLIBC6-LINUXLIBC6 confirms it.
The dash appears when the host and target are different.
The first platform is the host, the second is the target.

 
 - Jay

From wagner at elegosoft.com  Wed Nov 12 08:18:55 2008
From: wagner at elegosoft.com (Olaf Wagner)
Date: Wed, 12 Nov 2008 08:18:55 +0100
Subject: [M3devel] Still can't bootstrap cm3
In-Reply-To: <4919ED6D.6000700@wichita.edu>
References: <4919ED6D.6000700@wichita.edu>
Message-ID: <20081112081855.alr1p7k4ggw0k4k0@mail.elegosoft.com>

Quoting "Rodney M. Bates" <rodney.bates at wichita.edu>:

> We still have serious problems with the cm3 system building.
> I have again gotten into a big hampster wheel of several different
> failures, and still can't rebuild a working compiler.
> I have gone through enough steps that I am not just positive
> I have remembered it all right, but it was something like this:

Rodney,

it's not easy to understand what has gone wrong at this point.
Usually, if you have a working system, cm3/scripts/upgrade.sh
should be able to bootstrap to a new set of sources. This is tested
every day in the tinderbox release tests, too. As the last builds seem
to have succeeded, I don't think there is a real problem in our
code base currently. You may want to have a look at test_build_current
in cm3/scripts/regression/defs.sh to see what's done in the regression
builds.

I'd suggest you either start again from the last release or
one of the latest snapshots available from the build machine.
I assume you are not building a cross compiler, but just trying
to perform an upgrade.

Daily snapshots are available from
http://www.opencm3.net/snaps/snapshot-index.html

I hope this helps,

Olaf

> I started with ./do-cm3-front.sh, ./do-cm3-front.sh -ship, and
> ./install-cm3-compiler.sh upgrade.  That left me with a compiler
> that always failed with:
>
> ------------------------------------------------------------------------------------------------------------
>
> rodney at yellowstone:~/proj/m3/cm3-new/cm3/m3-sys/m3middle$ cm3 -build
> --- building in LINUXLIBC6 ---
>
> ignoring ../src/m3overrides
>
> "/usr/local/cm3/pkg/libm3/src/bundleintf/bundle-ov.tmpl", line 4: quake
> runtime error: undefined variable: ROOT
>
> --procedure--  -line-  -file---
> _import_template       4
> /usr/local/cm3/pkg/libm3/src/bundleintf/bundle-ov.tmpl
> _import_template       4
> /usr/local/cm3/pkg/libm3/src/bundleintf/bundle-ov.tmpl
> import            274  /usr/local/cm3/pkg/libm3/LINUXLIBC6/.M3EXPORTS
> include_dir        11
> /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3middle/src/m3makefile
>                     4
> /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3middle/LINUXLIBC6/m3make.args
>
> Fatal Error: package build failed
>
> -----------------------------------------------------------------------------------------------------------
>
> Then I tried rebuilding using various do-cm3-* scripts, including after
> supplying argument realclean (which, I have learned many times in the
> past, is essential to avoid all sorts of problems), with the same results.
> Somewhere in here, I started getting the bad version stamp failures too.
> Eventually, trying to rebootstrap with ./upgrade.sh, and also manually,
> a package at a time, I was consistently getting:
>
> --------------------------------------------------------------------------------------------------------
>
> rodney at yellowstone:~/proj/m3/cm3-new/cm3/m3-sys/m3middle$ cm3 -build
> --- building in LINUXLIBC6 ---
>
> ignoring ../src/m3overrides
>
> "/usr/local/cm3/pkg/libm3/LINUXLIBC6/.M3EXPORTS", line 4: quake runtime
> error: unable to open
> "/home/rodney/proj/m3/cm3-new/cm3/m3-libs/m3core/LINUXLIBC6/.M3EXPORTS"
> for reading
>
> --procedure--  -line-  -file---
> import_version      --  <builtin>
> import              4  /usr/local/cm3/pkg/libm3/LINUXLIBC6/.M3EXPORTS
> include_dir        11
> /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3middle/src/m3makefile
>                     4
> /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3middle/LINUXLIBC6/m3make.args
>
> Fatal Error: package build failed
>
> -----------------------------------------------------------------------------------------------------------
>
> Perhaps it was the realclean that deleted all the .M3EXPORTS files.
> ../boot_cm3-core.sh LINUXLIBC6 gave this failure:
>
> -----------------------------------------------------------------------------------------------------------
>
> rodney at yellowstone:~/proj/m3/cm3-new/cm3/scripts$ ./boot-cm3-core.sh
> please specify a cross compilation target platform
> rodney at yellowstone:~/proj/m3/cm3-new/cm3/scripts$ ./boot-cm3-core.sh
> LINUXLIBC6
> /home/rodney/proj/m3/cm3-new/cm3/scripts/pkgmap.sh -c "cm3 -build
> -override -DROOT='/home/rodney/proj/m3/cm3-new/cm3'
> -DCM3_VERSION_TEXT='d5.7.0' -DCM3_VERSION_NUMBER='050700'
> -DCM3_LAST_CHANGED='2008-03-16' -DM3_BOOTSTRAP=TRUE -keep" m3core libm3
> sysutils m3middle m3objfile m3linker m3back m3front m3quake cm3
> m3scanner m3tools m3cgcat m3cggen m3bundle bitvector digraph
> parseparams realgeometry set slisp sortedtableextras table-list
> tempfiles
> === package /home/rodney/proj/m3/cm3-new/cm3/m3-libs/m3core ===
>  +++ cm3 -build -override -DROOT='/home/rodney/proj/m3/cm3-new/cm3'
> -DCM3_VERSION_TEXT='d5.7.0' -DCM3_VERSION_NUMBER='050700'
> -DCM3_LAST_CHANGED='2008-03-16' -DM3_BOOTSTRAP=TRUE -keep +++
> --- building in LINUXLIBC6 ---
>
> new source -> compiling RTHooks.i3
> /bin/sh:
> /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3cc/LINUXLIBC6-LINUXLIBC6/cm3cg: not
> found
>   m3_backend => 127
> m3cc (aka cm3cg) failed compiling: RTHooks.ic
>
> ---------------------------------------------------------------------------------------------------------
>
> Fortunately, an earlier ./upgrade.sh had built a cm3cg (though it had failed
> on all the Modula-3 code).  I got around the above by putting in a symbolic
> link from cm3/m3-sys/m3cc/LINUXLIBC6-LINUXLIBC6 to   
> cm3/m3-sys/m3cc/LINUXLIBC6,
> so it could find cm3cg.
>
> Then I went through the package-at-a-time order Tony had given in an old post
> to the m3devel list.  That had other problems.  Running cm3 in the package
> directory failed with the undefined ROOT problem, but do-pkg.sh would do the
> build.  But do-pkg.sh doesn't appear to do a ship, so I had to do that in
> each package with cm3 -ship.
>
> After building and shipping m3 middle and m3front, package cm3   
> itself wouldn't
> link, wanting libraries for m3back, m3objfile, m3quake, sysutils,
> m3linker, libm3,
> and m3core.  I skipped it and m3gc*, which don't exist now, and went
> through the
> rest of the order, which completed without failures building.  But that gave
> me a compiler that fails with a new problem I have never seen before:
>
> ---------------------------------------------------------------------------------------------------------
>
> rodney at yellowstone:~/proj/m3/cm3-new/cm3/scripts$
> ./install-cm3-compiler.sh upgrade
> cp_if: /usr/local/cm3/bin/cm3 and /usr/local/cm3/bin/cm3-d5.7.0 identical
> cp_if: /usr/local/cm3/bin/cm3cg and /usr/local/cm3/bin/cm3cg-d5.7.0 identical
>
>
> ***
> *** runtime error:
> ***    A compile-time type is missing.
> ***    file "../src/Utils.m3"
> ***
>
> Aborted (core dumped)
> cp /home/rodney/proj/m3/cm3-new/cm3/m3-sys/cm3/LINUXLIBC6/cm3
> /usr/local/cm3/bin/cm3-
> cp /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3cc/LINUXLIBC6/cm3cg
> /usr/local/cm3/bin/cm3cg-
> cp /usr/local/cm3/bin/cm3- /usr/local/cm3/bin/cm3
> cp /usr/local/cm3/bin/cm3cg- /usr/local/cm3/bin/cm3cg
>
> ------------------------------------------------------------------------------------------------------------
>
> Going back to an older cm3 makes that go away and makes the ROOT problem
> recur.  Working around that with cm3   
> -DROOT="/home/rodney/proj/m3/cm3-new/cm3"
> get the compiler to run, but it doesn't do anything, because it thinks
> nothing needs to be recompiled.  I could delete stuff, but I fear that would
> just start me around another iteration of the same thing.
>
> I don't believe I have ever gotten a bootstrap to work.  I think I   
> have always
> had to download a recent binary and go from there.
>
> Any help would be greatly appreciated.
>
> Rodney Bates



-- 
Olaf Wagner -- elego Software Solutions GmbH
                Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany
phone: +49 30 23 45 86 96  mobile: +49 177 2345 869  fax: +49 30 23 45 86 95
    http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin
Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194



From jay.krell at cornell.edu  Wed Nov 12 08:57:31 2008
From: jay.krell at cornell.edu (Jay)
Date: Wed, 12 Nov 2008 07:57:31 +0000
Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
In-Reply-To: <B9A95C1E-B873-40A9-A045-CD38E32CA5D0@cs.purdue.edu>
References: <COL101-W3DB0BA622C4D31A82B602E6180@phx.gbl>
	<C47F6B5A-F154-4485-B839-96185B59E3B6@cs.purdue.edu>
	<COL101-W2CFF8666D1486BD514159E61B0@phx.gbl> 
	<B9A95C1E-B873-40A9-A045-CD38E32CA5D0@cs.purdue.edu> 
Message-ID: <BAY116-W5161CD049D8CA0E7ABAE4AE6140@phx.gbl>


Target.Little_endian is used to drive some target-specificity in the IR.
   I guess, after all, that is why you have to specify it.
 
The IR for RTLinker.m3 is identical for MIPS64_OPENBSD and SPARC64_SOLARIS, once I set them both to unaligned_procedures, which they both need. (Tangent, I ran m3cgcat on the .ms file a few times instead of the .mc file; that makes it crash.)
 
I can't claim SPARC64_SOLARIS is any paragon of working and correct though, by far.
I should also be able to compare with SPARC64_OPENBSD, I forgot about that, and didn't have an m3cg.
I only ran SPARC64_SOLARIS far enough to discover it needed unaligned procedures.
I think I should fork linux-common and solaris-common off of the minimal 29k openbsd-common, rather than worry about all the stuff in solaris-2-x and linux-libc, over 100k each. Then SPARC64_SOLARIS could/would be in better shape probably.
 
MIPS64_OPENBSD cm3 gets further with unaligned_procedures, along with the parse.c fix -- it gets to a bus error in RunMainBody.
Again an alignment problem, though I think maybe memory memory trashing.
The fault occurs here:
 
    desc_offset := init_depth * ADRSIZE (InitDesc);
      desc := init_stack + desc_offset;      init_depth := desc.low_link; << 
  InitDesc = RECORD    module   : RT0.ModulePtr;    low_link : INTEGER;  END;
desc_offset appears to not be a multiple of 4.
 
I will look into that.
 
Long story short -- consider again the parse.c change?
I admit I still haven't debugged it (or rather, the code that consumes the data produced by it).
 
 - Jay



From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduCC: m3devel at elegosoft.comSubject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Sun, 9 Nov 2008 11:10:54 +0000

I agree. I am suspicious too. I will investigate more. Thanks.  - Jay



From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 9 Nov 2008 10:01:58 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
Aren't the integer values supposed to be written endian-independently in the IR?  I am suspicious about these differences.




On 9 Nov 2008, at 01:25, Jay wrote:

Thanks for the tip/direction. Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the same" but for two systematic diffs:  - AMD64_LINUX has a bunch of added calls to check_nil.   This is a "bug" in it, inoptimal but ok code, because "first readable address" is 0. It should probably be 4k. I should write a program that iterates starting at 0 to find the value.  (or better yet, cm3 should have a -configure option that does this? -- for configuration values for which there is a safe default; in general, imho, the targets should be auto configured via C, assuming there is a native development environment available).  - some integers are different (again, systematically)  AMD64_LINUX: init_int  656 13 Int.64 init_chars  664 "    typecode " init_int  680 2 Int.64 init_var  688 v.1 0 init_int  696 13 Int.64 init_chars  704 "    typeid   " init_int  720 2 Int.64 init_var  728 v.1 0 init_int  736 13 Int.64 init_chars  744 "RunMainBody: " init_int  760 2 Int.64 init_var  768 v.1 0 init_int  776 19 Int.64 init_chars  784 "RunMainBody: exec: " init_int  808 2 Int.64 init_var  816 v.1 0 init_int  824 17 Int.64 init_chars  832 "ExpandInitStack: "MIPS64_OPENBSD:  init_chars  664 "    typecode " init_int  680 8796093022208 Int.64 init_var  688 v.1 0 init_int  696 13 Int.64 init_chars  704 "    typeid   " init_int  720 8796093022208 Int.64 init_var  728 v.1 0 init_int  736 13 Int.64 init_chars  744 "RunMainBody: " init_int  760 8796093022208 Int.64 init_var  768 v.1 0 init_int  776 19 Int.64 init_chars  784 "RunMainBody: exec: " init_int  808 8796093022208 Int.64 init_var  816 v.1 0This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where little endian has 2. I don't know what these numbers represent.I should look at other systems -- preferably a working 64 bit big endian system, if there are any. Could be flags/type with text literals? (Btw, I always leave the TextLiteral.i3 change on my system, lowering the maximum literal length to around 4gig even on 64 bit targets, so 32 bit can bootstrap 64 bit...need to fix the compiler here to do target arithmetic instead of host arithmetic...) Thanks, - Jay

From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sat, 8 Nov 2008 22:55:54 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Have you checked to make sure that the pre-gcc IR makes sense?  Does it see these values as 32-bit or 64-bit?  The reason I ask is that there may be something wrong in your m3middle target specs that cause bad IR.  Once we know the IR looks right then we can try and fix the problem.



Antony Hosking | Associate Professor | Computer Science | Purdue University
305 N. University Street | West Lafayette | IN 47907 | USA
Office +1 765 494 6001 | Mobile +1 765 427 5484


On 8 Nov 2008, at 15:45, Jay wrote:

truncated as usual...along with losing a lot of whitespace..Let's see...Tony, how about switching these #if's?I explain some of why below.static voidm3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){ tree val;#if 0 if (o != 0 || TREE_TYPE (v) != dst_t) {   v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),    bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) {   v = m3_build1 (ADDR_EXPR, t_addr, v);   v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));   v = m3_build1 (INDIRECT_REF, dst_t,    m3_cast (build_pointer_type (dst_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) {   val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP ();}m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){#if 0 if (o != 0 || TREE_TYPE (v) != src_t) {   v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),    bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) {   v = m3_build1 (ADDR_EXPR, t_addr, v);   v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));   v = m3_build1 (INDIRECT_REF, src_t,    m3_cast (build_pointer_type (src_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) {   v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v);}I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).In an early version it seems that many loads/stores of global variables is wrong.The "first" code affected is RTLinker_ExpandModuleTable.PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS;  n_bytes: INTEGER; BEGIN   IF (modules = NIL) THEN     (* first time... *)     max_modules := InitialTableSize;     modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr));     IF (modules = NIL) THEN Cstdlib.abort (); END;   ELSE     n_bytes := max_modules * BYTESIZE (RT0.ModulePtr);     new_mods := Cstdlib.malloc (n_bytes + n_bytes);     IF (new_mods = NIL) THEN Cstdlib.abort (); END;     EVAL Cstring.memcpy (new_mods, modules, n_bytes);     Cstdlib.free (modules);     modules := new_mods;     INC (max_modules, max_modules);   END; END ExpandModuleTable;It should read back modules as NIL.What happens is it reads back modules as not NIL,and then max_modules should also be zero, but is "large", leading to malloc to fail,and abort called.The output of cm3cg -y is reasonable.It indicates offset loads.What is incorrect is that in many loads and stores, but not all, theoffsets are lost.(923) begin_procedure procedure RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable(924) set_source_line source line  208(925) load_nil(926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb(927) set_source_line source line  207(928) set_source_line source line  210(929) load_nil(930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xbHere is the incorrect code:.globl RTLinker__ExpandModuleTable.stabd 46,0,0.stabn 68,0,206,.LM108.LM108:.LFBB8:.set nomips16.ent RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable:.frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerld $2,0($2)      << WRONGbne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $3,MM_RTLinkerld $2,128($3)andi $2,$2,0x0ori $2,$2,0x1f4sd $2,128($3).stabn 68,0,213,.LM112.LM112:li $4,4000   # 0xfa0jal mallocsd $2,0($fp)ld $4,0($fp)dla $3,MM_RTLinkerld $2,136($3)andi $2,$2,0x0or $2,$4,$2sd $2,136($3)     << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerld $2,0($2)       << WRONG bne $2,$0,.L65jal abortb .L65.L61:.stabn 68,0,216,.LM114.LM114:dla $2,MM_RTLinkerld $2,0($2)        << WRONG dsll $2,$2,3sd $2,8($fp).stabn 68,0,217,.LM115.LM115:ld $3,8($fp)ld $2,8($fp)daddu $2,$3,$2move $4,$2jal mallocsd $2,0($fp)ld $2,0($fp)sd $2,16($fp).stabn 68,0,218,.LM116.LM116:ld $2,16($fp)bne $2,$0,.L64jal abort.L64:.stabn 68,0,219,.LM117.LM117:ld $4,16($fp)dla $2,MM_RTLinkerld $2,0($2)          << WRONGld $3,8($fp)move $5,$2move $6,$3jal memcpysd $2,0($fp).stabn 68,0,220,.LM118.LM118:dla $2,MM_RTLinkerld $2,0($2)           << WRONGmove $4,$2jal free.stabn 68,0,221,.LM119.LM119:ld $4,16($fp)dla $3,MM_RTLinkerld $2,136($3)         << OKandi $2,$2,0x0or $2,$4,$2sd $2,136($3)         << OK .stabn 68,0,222,.LM120.LM120:dla $2,MM_RTLinkerld $3,0($2)           << WRONGdla $2,MM_RTLinkerld $2,0($2)           << WRONGdaddu $4,$3,$2dla $3,MM_RTLinkerld $2,128($3)         << OKandi $2,$2,0x0or $2,$2,$4sd $2,128($3)         << OK.L65:.LBE9:.stabn 68,0,224,.LM121.LM121:move $sp,$fpld $31,56($sp)ld $fp,48($sp)ld $28,40($sp)daddiu $sp,$sp,64j $31.end RTLinker__ExpandModuleTableHere is corrected code:RTLinker__ExpandModuleTable:.frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerdaddiu $2,$2,136     << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2)bne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $2,MM_RTLinkerdaddiu $3,$2,128     << CORRECT li $2,500   # 0x1f4sd $2,0($3).stabn 68,0,213,.LM112.LM112:li $4,4000   # 0xfa0jal mallocsd $2,0($fp)dla $2,MM_RTLinkerdaddiu $3,$2,136      << CORRECT ld $2,0($fp)sd $2,0($3).stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerdaddiu $2,$2,136      << CORRECT ld $2,0($2)bne $2,$0,.L65jal abortb .L65.L61:I still dump core with this change, but I get much further.I have not tested this change at all otherwise -- i.e. on working platforms.I have not debugged through the gcc/m3cg code to see whythe offsets are only sometimes lost, or why other platformshave no problem. Is MIPS's "global pointer" unusual these days?I know SPARC has similar, and all the non-x86, non-AMD64 NT platformshad similar, but that's essentially nothing now (I'm not counting IA64).I know Linux has PLT (procedure linkage table) and GOT (global offset table).Same thing?I haven't looked much at other MIPS ports yet.http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8is what made it look about like it does -- in particularing using BIT_FIELD_REF.1.7:static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */    tree v;    int o;    tree src_t, dest_t;    m3_type src_T, dest_T;{ if (o == 0 && TREE_TYPE (v) == src_t) {   EXPR_PUSH (v); } else {   tree adr = m3_build1 (ADDR_EXPR, t_addr, v);   if (o != 0) {     adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT));   }   EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t,                         m3_cast (build_pointer_type (src_t), adr))); }#if 1 if (src_T != dest_T) {   EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) {   const char *name = "noname";   if (v != 0 && DECL_NAME(v) != 0) {     name = IDENTIFIER_POINTER(DECL_NAME(v));   }   fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n", name,           o, src_T, dest_T); }#else if (src_T != dest_T) {   if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) {     EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1));   } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) {     EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1));   } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) {     /* okay */   } else {     fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n",                 src_T, dest_T);   } }#endif}static voidm3_store (v, o, src_t, dest_t)  /* GCC32OK */    tree v;    int o;    tree src_t, dest_t;{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) {   rhs = EXPR_REF (-1); } else {   rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) {   lhs = v; } else {   tree f = make_node (FIELD_DECL);   TREE_TYPE (f) = dest_t;   DECL_ALIGN (f) = TYPE_ALIGN (dest_t);   DECL_SIZE (f) = TYPE_SIZE (dest_t);   DECL_MODE (f) = TYPE_MODE (dest_t);   DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT);   DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT);   DECL_FIELD_CONTEXT (f) = TREE_TYPE (v);   lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); }#if 1   expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);#else if (dest_t ==  NULL) {   expand_assignment (lhs, rhs, 0, 0); } else {   expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); }#endif EXPR_POP ();}1.8:static voidm3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */{ if (o == 0 && TREE_TYPE (v) == src_t) {   EXPR_PUSH (v); } else {   EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),    bitsize_int (o))); } if (src_T != dst_T) {   EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) {   const char *name = "noname";   if (v != 0 && DECL_NAME(v) != 0) {     name = IDENTIFIER_POINTER(DECL_NAME(v));   }   fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n", name,    o, src_T, dst_T); }}static voidm3_store (tree v, int o, tree src_t, tree dst_t)  /* GCC32OK */{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) {   rhs = EXPR_REF (-1); } else {   rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) {   lhs = v; } else {   lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),      bitsize_int (o)); } if (src_t != dst_t) {   rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP ();}- Jay<AMD64_LINUX_RTLinker.mc><MIPS64_OPENBSD_RTLinker.mc>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081112/afd2a674/attachment.html>

From jay.krell at cornell.edu  Wed Nov 12 09:05:03 2008
From: jay.krell at cornell.edu (Jay)
Date: Wed, 12 Nov 2008 08:05:03 +0000
Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
In-Reply-To: <BAY116-W5161CD049D8CA0E7ABAE4AE6140@phx.gbl>
References: <COL101-W3DB0BA622C4D31A82B602E6180@phx.gbl>
	<C47F6B5A-F154-4485-B839-96185B59E3B6@cs.purdue.edu>
	<COL101-W2CFF8666D1486BD514159E61B0@phx.gbl> 
	<B9A95C1E-B873-40A9-A045-CD38E32CA5D0@cs.purdue.edu>  
	<BAY116-W5161CD049D8CA0E7ABAE4AE6140@phx.gbl>
Message-ID: <BAY121-W536D2BD8E7F1E500416F5FE6140@phx.gbl>


ps: it takes over 20 seconds for MIPS64_OPENBSD to crash in RunMainBody, which makes me suspect the platform is impractically slow.
I can only hope that optimization and/or not crashing makes it much faster.
The bootstrap build -- assemble a bunch, compile a little C, link, is also slow though.
System is 200MHz+ and 256meg RAM though, ought to suffice..
 
- Jay



From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduDate: Wed, 12 Nov 2008 07:57:31 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??

Target.Little_endian is used to drive some target-specificity in the IR.   I guess, after all, that is why you have to specify it. The IR for RTLinker.m3 is identical for MIPS64_OPENBSD and SPARC64_SOLARIS, once I set them both to unaligned_procedures, which they both need. (Tangent, I ran m3cgcat on the .ms file a few times instead of the .mc file; that makes it crash.) I can't claim SPARC64_SOLARIS is any paragon of working and correct though, by far.I should also be able to compare with SPARC64_OPENBSD, I forgot about that, and didn't have an m3cg.I only ran SPARC64_SOLARIS far enough to discover it needed unaligned procedures.I think I should fork linux-common and solaris-common off of the minimal 29k openbsd-common, rather than worry about all the stuff in solaris-2-x and linux-libc, over 100k each. Then SPARC64_SOLARIS could/would be in better shape probably. MIPS64_OPENBSD cm3 gets further with unaligned_procedures, along with the parse.c fix -- it gets to a bus error in RunMainBody.Again an alignment problem, though I think maybe memory memory trashing.The fault occurs here:     desc_offset := init_depth * ADRSIZE (InitDesc);      desc := init_stack + desc_offset;      init_depth := desc.low_link; <<   InitDesc = RECORD    module   : RT0.ModulePtr;    low_link : INTEGER;  END;desc_offset appears to not be a multiple of 4. I will look into that. Long story short -- consider again the parse.c change?I admit I still haven't debugged it (or rather, the code that consumes the data produced by it).  - Jay



From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduCC: m3devel at elegosoft.comSubject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Sun, 9 Nov 2008 11:10:54 +0000

I agree. I am suspicious too. I will investigate more. Thanks.  - Jay



From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 9 Nov 2008 10:01:58 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
Aren't the integer values supposed to be written endian-independently in the IR?  I am suspicious about these differences.




On 9 Nov 2008, at 01:25, Jay wrote:

Thanks for the tip/direction. Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the same" but for two systematic diffs:  - AMD64_LINUX has a bunch of added calls to check_nil.   This is a "bug" in it, inoptimal but ok code, because "first readable address" is 0. It should probably be 4k. I should write a program that iterates starting at 0 to find the value.  (or better yet, cm3 should have a -configure option that does this? -- for configuration values for which there is a safe default; in general, imho, the targets should be auto configured via C, assuming there is a native development environment available).  - some integers are different (again, systematically)  AMD64_LINUX: init_int  656 13 Int.64 init_chars  664 "    typecode " init_int  680 2 Int.64 init_var  688 v.1 0 init_int  696 13 Int.64 init_chars  704 "    typeid   " init_int  720 2 Int.64 init_var  728 v.1 0 init_int  736 13 Int.64 init_chars  744 "RunMainBody: " init_int  760 2 Int.64 init_var  768 v.1 0 init_int  776 19 Int.64 init_chars  784 "RunMainBody: exec: " init_int  808 2 Int.64 init_var  816 v.1 0 init_int  824 17 Int.64 init_chars  832 "ExpandInitStack: "MIPS64_OPENBSD:  init_chars  664 "    typecode " init_int  680 8796093022208 Int.64 init_var  688 v.1 0 init_int  696 13 Int.64 init_chars  704 "    typeid   " init_int  720 8796093022208 Int.64 init_var  728 v.1 0 init_int  736 13 Int.64 init_chars  744 "RunMainBody: " init_int  760 8796093022208 Int.64 init_var  768 v.1 0 init_int  776 19 Int.64 init_chars  784 "RunMainBody: exec: " init_int  808 8796093022208 Int.64 init_var  816 v.1 0This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where little endian has 2. I don't know what these numbers represent.I should look at other systems -- preferably a working 64 bit big endian system, if there are any. Could be flags/type with text literals? (Btw, I always leave the TextLiteral.i3 change on my system, lowering the maximum literal length to around 4gig even on 64 bit targets, so 32 bit can bootstrap 64 bit...need to fix the compiler here to do target arithmetic instead of host arithmetic...) Thanks, - Jay

From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sat, 8 Nov 2008 22:55:54 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Have you checked to make sure that the pre-gcc IR makes sense?  Does it see these values as 32-bit or 64-bit?  The reason I ask is that there may be something wrong in your m3middle target specs that cause bad IR.  Once we know the IR looks right then we can try and fix the problem.



Antony Hosking | Associate Professor | Computer Science | Purdue University
305 N. University Street | West Lafayette | IN 47907 | USA
Office +1 765 494 6001 | Mobile +1 765 427 5484


On 8 Nov 2008, at 15:45, Jay wrote:

truncated as usual...along with losing a lot of whitespace..Let's see...Tony, how about switching these #if's?I explain some of why below.static voidm3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){ tree val;#if 0 if (o != 0 || TREE_TYPE (v) != dst_t) {   v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),    bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) {   v = m3_build1 (ADDR_EXPR, t_addr, v);   v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));   v = m3_build1 (INDIRECT_REF, dst_t,    m3_cast (build_pointer_type (dst_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) {   val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP ();}m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){#if 0 if (o != 0 || TREE_TYPE (v) != src_t) {   v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),    bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) {   v = m3_build1 (ADDR_EXPR, t_addr, v);   v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));   v = m3_build1 (INDIRECT_REF, src_t,    m3_cast (build_pointer_type (src_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) {   v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v);}I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).In an early version it seems that many loads/stores of global variables is wrong.The "first" code affected is RTLinker_ExpandModuleTable.PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS;  n_bytes: INTEGER; BEGIN   IF (modules = NIL) THEN     (* first time... *)     max_modules := InitialTableSize;     modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr));     IF (modules = NIL) THEN Cstdlib.abort (); END;   ELSE     n_bytes := max_modules * BYTESIZE (RT0.ModulePtr);     new_mods := Cstdlib.malloc (n_bytes + n_bytes);     IF (new_mods = NIL) THEN Cstdlib.abort (); END;     EVAL Cstring.memcpy (new_mods, modules, n_bytes);     Cstdlib.free (modules);     modules := new_mods;     INC (max_modules, max_modules);   END; END ExpandModuleTable;It should read back modules as NIL.What happens is it reads back modules as not NIL,and then max_modules should also be zero, but is "large", leading to malloc to fail,and abort called.The output of cm3cg -y is reasonable.It indicates offset loads.What is incorrect is that in many loads and stores, but not all, theoffsets are lost.(923) begin_procedure procedure RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable(924) set_source_line source line  208(925) load_nil(926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb(927) set_source_line source line  207(928) set_source_line source line  210(929) load_nil(930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xbHere is the incorrect code:.globl RTLinker__ExpandModuleTable.stabd 46,0,0.stabn 68,0,206,.LM108.LM108:.LFBB8:.set nomips16.ent RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable:.frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerld $2,0($2)      << WRONGbne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $3,MM_RTLinkerld $2,128($3)andi $2,$2,0x0ori $2,$2,0x1f4sd $2,128($3).stabn 68,0,213,.LM112.LM112:li $4,4000   # 0xfa0jal mallocsd $2,0($fp)ld $4,0($fp)dla $3,MM_RTLinkerld $2,136($3)andi $2,$2,0x0or $2,$4,$2sd $2,136($3)     << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerld $2,0($2)       << WRONG bne $2,$0,.L65jal abortb .L65.L61:.stabn 68,0,216,.LM114.LM114:dla $2,MM_RTLinkerld $2,0($2)        << WRONG dsll $2,$2,3sd $2,8($fp).stabn 68,0,217,.LM115.LM115:ld $3,8($fp)ld $2,8($fp)daddu $2,$3,$2move $4,$2jal mallocsd $2,0($fp)ld $2,0($fp)sd $2,16($fp).stabn 68,0,218,.LM116.LM116:ld $2,16($fp)bne $2,$0,.L64jal abort.L64:.stabn 68,0,219,.LM117.LM117:ld $4,16($fp)dla $2,MM_RTLinkerld $2,0($2)          << WRONGld $3,8($fp)move $5,$2move $6,$3jal memcpysd $2,0($fp).stabn 68,0,220,.LM118.LM118:dla $2,MM_RTLinkerld $2,0($2)           << WRONGmove $4,$2jal free.stabn 68,0,221,.LM119.LM119:ld $4,16($fp)dla $3,MM_RTLinkerld $2,136($3)         << OKandi $2,$2,0x0or $2,$4,$2sd $2,136($3)         << OK .stabn 68,0,222,.LM120.LM120:dla $2,MM_RTLinkerld $3,0($2)           << WRONGdla $2,MM_RTLinkerld $2,0($2)           << WRONGdaddu $4,$3,$2dla $3,MM_RTLinkerld $2,128($3)         << OKandi $2,$2,0x0or $2,$2,$4sd $2,128($3)         << OK.L65:.LBE9:.stabn 68,0,224,.LM121.LM121:move $sp,$fpld $31,56($sp)ld $fp,48($sp)ld $28,40($sp)daddiu $sp,$sp,64j $31.end RTLinker__ExpandModuleTableHere is corrected code:RTLinker__ExpandModuleTable:.frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerdaddiu $2,$2,136     << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2)bne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $2,MM_RTLinkerdaddiu $3,$2,128     << CORRECT li $2,500   # 0x1f4sd $2,0($3).stabn 68,0,213,.LM112.LM112:li $4,4000   # 0xfa0jal mallocsd $2,0($fp)dla $2,MM_RTLinkerdaddiu $3,$2,136      << CORRECT ld $2,0($fp)sd $2,0($3).stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerdaddiu $2,$2,136      << CORRECT ld $2,0($2)bne $2,$0,.L65jal abortb .L65.L61:I still dump core with this change, but I get much further.I have not tested this change at all otherwise -- i.e. on working platforms.I have not debugged through the gcc/m3cg code to see whythe offsets are only sometimes lost, or why other platformshave no problem. Is MIPS's "global pointer" unusual these days?I know SPARC has similar, and all the non-x86, non-AMD64 NT platformshad similar, but that's essentially nothing now (I'm not counting IA64).I know Linux has PLT (procedure linkage table) and GOT (global offset table).Same thing?I haven't looked much at other MIPS ports yet.http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8is what made it look about like it does -- in particularing using BIT_FIELD_REF.1.7:static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */    tree v;    int o;    tree src_t, dest_t;    m3_type src_T, dest_T;{ if (o == 0 && TREE_TYPE (v) == src_t) {   EXPR_PUSH (v); } else {   tree adr = m3_build1 (ADDR_EXPR, t_addr, v);   if (o != 0) {     adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT));   }   EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t,                         m3_cast (build_pointer_type (src_t), adr))); }#if 1 if (src_T != dest_T) {   EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) {   const char *name = "noname";   if (v != 0 && DECL_NAME(v) != 0) {     name = IDENTIFIER_POINTER(DECL_NAME(v));   }   fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n", name,           o, src_T, dest_T); }#else if (src_T != dest_T) {   if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) {     EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1));   } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) {     EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1));   } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) {     /* okay */   } else {     fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n",                 src_T, dest_T);   } }#endif}static voidm3_store (v, o, src_t, dest_t)  /* GCC32OK */    tree v;    int o;    tree src_t, dest_t;{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) {   rhs = EXPR_REF (-1); } else {   rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) {   lhs = v; } else {   tree f = make_node (FIELD_DECL);   TREE_TYPE (f) = dest_t;   DECL_ALIGN (f) = TYPE_ALIGN (dest_t);   DECL_SIZE (f) = TYPE_SIZE (dest_t);   DECL_MODE (f) = TYPE_MODE (dest_t);   DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT);   DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT);   DECL_FIELD_CONTEXT (f) = TREE_TYPE (v);   lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); }#if 1   expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);#else if (dest_t ==  NULL) {   expand_assignment (lhs, rhs, 0, 0); } else {   expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); }#endif EXPR_POP ();}1.8:static voidm3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */{ if (o == 0 && TREE_TYPE (v) == src_t) {   EXPR_PUSH (v); } else {   EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),    bitsize_int (o))); } if (src_T != dst_T) {   EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) {   const char *name = "noname";   if (v != 0 && DECL_NAME(v) != 0) {     name = IDENTIFIER_POINTER(DECL_NAME(v));   }   fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n", name,    o, src_T, dst_T); }}static voidm3_store (tree v, int o, tree src_t, tree dst_t)  /* GCC32OK */{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) {   rhs = EXPR_REF (-1); } else {   rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) {   lhs = v; } else {   lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),      bitsize_int (o)); } if (src_t != dst_t) {   rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP ();}- Jay<AMD64_LINUX_RTLinker.mc><MIPS64_OPENBSD_RTLinker.mc>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081112/c2cc1a47/attachment.html>

From rcoleburn at scires.com  Wed Nov 12 19:49:19 2008
From: rcoleburn at scires.com (Randy Coleburn)
Date: Wed, 12 Nov 2008 13:49:19 -0500
Subject: [M3devel] [M3commit] CVS Update: cm3
In-Reply-To: <20081112152142.02AEA10D5970@birch.elegosoft.com>
References: <20081112152142.02AEA10D5970@birch.elegosoft.com>
Message-ID: <491ADDBA.1E75.00D7.1@scires.com>

Rodney:
 
Is this change simply to make PM3 and CM3 more compatible wrt pickles, or does it alter current behavior for folks who only use CM3 ? (i.e., will it result in a change to the way CM3 pickles are represented that may affect current CM3 users?)
 
Regards,
Randy

>>> Rodney M. Bates <rodney at elego.de> 11/12/2008 4:21 PM >>>
CVSROOT:/usr/cvs
Changes by:rodney at birch.08/11/12 16:21:41

Modified files:
cm3/m3-libs/libm3/src/pickle/ver2/: Pickle2.m3 

Log message:
Pickles in both PM3 and CM3 have some hard-coded
fingerprints for common types, but the values
are not consistent about byte order.

Fix CM2 pickle-reading code to accept values of types
NULL, ROOT, UNTRACED ROOT, ADDRESS, AND REFANY,
written by PM3 pickle-writing code.  Probably,
the latter two can't even happen in a pickle, unless
somebody wrote a custom special for them.

This still won't handle a more complex type that depends
on these types, e.g., an object that contains a field of
one of them.  This latter is probably not feasable, because
it would require "decrypting" fingerprints, and the hash
algorithm for fingerprints is lossy.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081112/456771f5/attachment.html>

From hosking at cs.purdue.edu  Wed Nov 12 21:46:53 2008
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Wed, 12 Nov 2008 20:46:53 +0000
Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
In-Reply-To: <BAY116-W5161CD049D8CA0E7ABAE4AE6140@phx.gbl>
References: <COL101-W3DB0BA622C4D31A82B602E6180@phx.gbl>
	<C47F6B5A-F154-4485-B839-96185B59E3B6@cs.purdue.edu>
	<COL101-W2CFF8666D1486BD514159E61B0@phx.gbl>
	<B9A95C1E-B873-40A9-A045-CD38E32CA5D0@cs.purdue.edu>
	<BAY116-W5161CD049D8CA0E7ABAE4AE6140@phx.gbl>
Message-ID: <80AE04BD-4819-4085-8EBE-5996E8E6DB93@cs.purdue.edu>

Jay, do you have access to an AMD64_DARWIN box?   I did get things  
working on that 64-bit platform.   Or is it an endianness issue you  
are dealing with.  In any case, I will not have a lot of bandwidth for  
doing much in the way of tackling this unless it becomes critical path  
for me soon.

Antony Hosking | Associate Professor | Computer Science | Purdue  
University
305 N. University Street | West Lafayette | IN 47907 | USA
Office +1 765 494 6001 | Mobile +1 765 427 5484



On 12 Nov 2008, at 07:57, Jay wrote:

> Target.Little_endian is used to drive some target-specificity in the  
> IR.
>    I guess, after all, that is why you have to specify it.
>
> The IR for RTLinker.m3 is identical for MIPS64_OPENBSD and  
> SPARC64_SOLARIS, once I set them both to unaligned_procedures, which  
> they both need. (Tangent, I ran m3cgcat on the .ms file a few times  
> instead of the .mc file; that makes it crash.)
>
> I can't claim SPARC64_SOLARIS is any paragon of working and correct  
> though, by far.
> I should also be able to compare with SPARC64_OPENBSD, I forgot  
> about that, and didn't have an m3cg.
> I only ran SPARC64_SOLARIS far enough to discover it needed  
> unaligned procedures.
> I think I should fork linux-common and solaris-common off of the  
> minimal 29k openbsd-common, rather than worry about all the stuff in  
> solaris-2-x and linux-libc, over 100k each. Then SPARC64_SOLARIS  
> could/would be in better shape probably.
>
> MIPS64_OPENBSD cm3 gets further with unaligned_procedures, along  
> with the parse.c fix -- it gets to a bus error in RunMainBody.
> Again an alignment problem, though I think maybe memory memory  
> trashing.
> The fault occurs here:
>
>     desc_offset := init_depth * ADRSIZE (InitDesc);
>
>       desc := init_stack + desc_offset;
>       init_depth := desc.low_link; <<
>
>   InitDesc = RECORD
>     module   : RT0.ModulePtr;
>     low_link : INTEGER;
>   END;
>
> desc_offset appears to not be a multiple of 4.
>
> I will look into that.
>
> Long story short -- consider again the parse.c change?
> I admit I still haven't debugged it (or rather, the code that  
> consumes the data produced by it).
>
>  - Jay
>
>
>
> From: jay.krell at cornell.edu
> To: hosking at cs.purdue.edu
> CC: m3devel at elegosoft.com
> Subject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
> Date: Sun, 9 Nov 2008 11:10:54 +0000
>
> I agree. I am suspicious too. I will investigate more. Thanks.
>
>  - Jay
>
>
>
>
> From: hosking at cs.purdue.edu
> To: jay.krell at cornell.edu
> Date: Sun, 9 Nov 2008 10:01:58 +0000
> CC: m3devel at elegosoft.com
> Subject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
>
>
>
> Aren't the integer values supposed to be written endian- 
> independently in the IR?  I am suspicious about these differences.
>
> On 9 Nov 2008, at 01:25, Jay wrote:
>
> Thanks for the tip/direction.
>
> Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the  
> same" but for two systematic diffs:
>
>  - AMD64_LINUX has a bunch of added calls to check_nil.
>    This is a "bug" in it, inoptimal but ok code, because "first  
> readable address" is 0. It should probably be 4k. I should write a  
> program that iterates starting at 0 to find the value.
>   (or better yet, cm3 should have a -configure option that does  
> this? -- for configuration values for which there is a safe default;  
> in general, imho, the targets should be auto configured via C,  
> assuming there is a native development environment available).
>
>  - some integers are different (again, systematically)
>
> AMD64_LINUX:
>  init_int  656 13 Int.64
>  init_chars  664 "    typecode "
>  init_int  680 2 Int.64
>  init_var  688 v.1 0
>  init_int  696 13 Int.64
>  init_chars  704 "    typeid   "
>  init_int  720 2 Int.64
>  init_var  728 v.1 0
>  init_int  736 13 Int.64
>  init_chars  744 "RunMainBody: "
>  init_int  760 2 Int.64
>  init_var  768 v.1 0
>  init_int  776 19 Int.64
>  init_chars  784 "RunMainBody: exec: "
>  init_int  808 2 Int.64
>  init_var  816 v.1 0
>  init_int  824 17 Int.64
>  init_chars  832 "ExpandInitStack: "
>
> MIPS64_OPENBSD:
>
>  init_chars  664 "    typecode "
>  init_int  680 8796093022208 Int.64
>  init_var  688 v.1 0
>  init_int  696 13 Int.64
>  init_chars  704 "    typeid   "
>  init_int  720 8796093022208 Int.64
>  init_var  728 v.1 0
>  init_int  736 13 Int.64
>  init_chars  744 "RunMainBody: "
>  init_int  760 8796093022208 Int.64
>  init_var  768 v.1 0
>  init_int  776 19 Int.64
>  init_chars  784 "RunMainBody: exec: "
>  init_int  808 8796093022208 Int.64
>  init_var  816 v.1 0
>
> This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where  
> little endian has 2.
>
> I don't know what these numbers represent.
> I should look at other systems -- preferably a working 64 bit big  
> endian system, if there are any.
>
> Could be flags/type with text literals?
>
> (Btw, I always leave the TextLiteral.i3 change on my system,  
> lowering the maximum literal length to around 4gig even on 64 bit  
> targets, so 32 bit can bootstrap 64 bit...need to fix the compiler  
> here to do target arithmetic instead of host arithmetic...)
>
> Thanks,
>  - Jay
>
>
> From: hosking at cs.purdue.edu
> To: jay.krell at cornell.edu
> Date: Sat, 8 Nov 2008 22:55:54 +0000
> CC: m3devel at elegosoft.com
> Subject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
>
> Have you checked to make sure that the pre-gcc IR makes sense?  Does  
> it see these values as 32-bit or 64-bit?  The reason I ask is that  
> there may be something wrong in your m3middle target specs that  
> cause bad IR.  Once we know the IR looks right then we can try and  
> fix the problem.
>
> Antony Hosking | Associate Professor | Computer Science | Purdue  
> University
> 305 N. University Street | West Lafayette | IN 47907 | USA
> Office +1 765 494 6001 | Mobile +1 765 427 5484
>
>
>
> On 8 Nov 2008, at 15:45, Jay wrote:
>
>
> truncated as usual...along with losing a lot of whitespace..
> Let's see...
>
>
> Tony, how about switching these #if's?
> I explain some of why below.
>
> static void
> m3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t,  
> m3_type dst_T)
> {
>  tree val;
> #if 0
>  if (o != 0 || TREE_TYPE (v) != dst_t) {
>    v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),
>     bitsize_int (o));
>  }
> #else
>  /* failsafe, but inefficient */
>  if (o != 0 || TREE_TYPE (v) != dst_t) {
>    v = m3_build1 (ADDR_EXPR, t_addr, v);
>    v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));
>    v = m3_build1 (INDIRECT_REF, dst_t,
>     m3_cast (build_pointer_type (dst_t), v));
>  }
> #endif
>  TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing  
> problems */
>  val = m3_cast (src_t, EXPR_REF (-1));
>  if (src_T != dst_T) {
>    val = m3_build1 (CONVERT_EXPR, dst_t, val);
>  }
>  add_stmt (build2 (MODIFY_EXPR, dst_t, v, val));
>  EXPR_POP ();
> }
> m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t,  
> m3_type dst_T)
> {
> #if 0
>  if (o != 0 || TREE_TYPE (v) != src_t) {
>    v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),
>     bitsize_int (o));
>  }
> #else
>  /* failsafe, but inefficient */
>  if (o != 0 || TREE_TYPE (v) != src_t) {
>    v = m3_build1 (ADDR_EXPR, t_addr, v);
>    v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));
>    v = m3_build1 (INDIRECT_REF, src_t,
>     m3_cast (build_pointer_type (src_t), v));
>  }
> #endif
>  TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing  
> problems */
>  if (src_T != dst_T) {
>    v = m3_build1 (CONVERT_EXPR, dst_t, v);
>  }
>  EXPR_PUSH (v);
> }
>
> I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).
> In an early version it seems that many loads/stores of global  
> variables is wrong.
> The "first" code affected is RTLinker_ExpandModuleTable.
>
> PROCEDURE ExpandModuleTable () =
>  CONST InitialTableSize = 500;
>  VAR new_mods: ADDRESS;  n_bytes: INTEGER;
>  BEGIN
>    IF (modules = NIL) THEN
>      (* first time... *)
>      max_modules := InitialTableSize;
>      modules := Cstdlib.malloc (InitialTableSize * BYTESIZE  
> (RT0.ModulePtr));
>      IF (modules = NIL) THEN Cstdlib.abort (); END;
>    ELSE
>      n_bytes := max_modules * BYTESIZE (RT0.ModulePtr);
>      new_mods := Cstdlib.malloc (n_bytes + n_bytes);
>      IF (new_mods = NIL) THEN Cstdlib.abort (); END;
>      EVAL Cstring.memcpy (new_mods, modules, n_bytes);
>      Cstdlib.free (modules);
>      modules := new_mods;
>      INC (max_modules, max_modules);
>    END;
>  END ExpandModuleTable;
>
> It should read back modules as NIL.
> What happens is it reads back modules as not NIL,
> and then max_modules should also be zero, but is "large", leading to  
> malloc to fail,
> and abort called.
>
> The output of cm3cg -y is reasonable.
> It indicates offset loads.
>
> What is incorrect is that in many loads and stores, but not all, the
> offsets are lost.
>
> (923) begin_procedure
>  procedure RTLinker__ExpandModuleTable
> RTLinker__ExpandModuleTable(924) set_source_line
>  source line  208
> (925) load_nil
> (926) store
>  store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb
> (927) set_source_line
>  source line  207
> (928) set_source_line
>  source line  210
> (929) load_nil
> (930) load
>  m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xb
>
> Here is the incorrect code:
>
> .globl RTLinker__ExpandModuleTable
> .stabd 46,0,0
> .stabn 68,0,206,.LM108
> .LM108:
> .LFBB8:
> .set nomips16
> .ent RTLinker__ExpandModuleTable
> RTLinker__ExpandModuleTable:
> .frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0
> .mask 0xd0000000,-8
> .fmask 0x00000000,0
> daddiu $sp,$sp,-64
> sd $31,56($sp)
> sd $fp,48($sp)
> sd $28,40($sp)
> move $fp,$sp
> lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
> daddu $28,$28,$25
> daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
> .LBB9:
> .stabn 68,0,208,.LM109
> .LM109:
> sd $0,16($fp)
> .stabn 68,0,210,.LM110
> .LM110:
> dla $2,MM_RTLinker
> ld $2,0($2)      << WRONG
> bne $2,$0,.L61
> .stabn 68,0,212,.LM111
> .LM111:
> dla $3,MM_RTLinker
> ld $2,128($3)
> andi $2,$2,0x0
> ori $2,$2,0x1f4
> sd $2,128($3)
> .stabn 68,0,213,.LM112
> .LM112:
> li $4,4000   # 0xfa0
> jal malloc
> sd $2,0($fp)
> ld $4,0($fp)
> dla $3,MM_RTLinker
> ld $2,136($3)
> andi $2,$2,0x0
> or $2,$4,$2
> sd $2,136($3)     << OK, offset seems low, but it agrees with  
> elsewhere
> .stabn 68,0,214,.LM113
> .LM113:
> dla $2,MM_RTLinker
> ld $2,0($2)       << WRONG
> bne $2,$0,.L65
> jal abort
> b .L65
> .L61:
> .stabn 68,0,216,.LM114
> .LM114:
> dla $2,MM_RTLinker
> ld $2,0($2)        << WRONG
> dsll $2,$2,3
> sd $2,8($fp)
> .stabn 68,0,217,.LM115
> .LM115:
> ld $3,8($fp)
> ld $2,8($fp)
> daddu $2,$3,$2
> move $4,$2
> jal malloc
> sd $2,0($fp)
> ld $2,0($fp)
> sd $2,16($fp)
> .stabn 68,0,218,.LM116
> .LM116:
> ld $2,16($fp)
> bne $2,$0,.L64
> jal abort
> .L64:
> .stabn 68,0,219,.LM117
> .LM117:
> ld $4,16($fp)
> dla $2,MM_RTLinker
> ld $2,0($2)          << WRONG
> ld $3,8($fp)
> move $5,$2
> move $6,$3
> jal memcpy
> sd $2,0($fp)
> .stabn 68,0,220,.LM118
> .LM118:
> dla $2,MM_RTLinker
> ld $2,0($2)           << WRONG
> move $4,$2
> jal free
> .stabn 68,0,221,.LM119
> .LM119:
> ld $4,16($fp)
> dla $3,MM_RTLinker
> ld $2,136($3)         << OK
> andi $2,$2,0x0
> or $2,$4,$2
> sd $2,136($3)         << OK
> .stabn 68,0,222,.LM120
> .LM120:
> dla $2,MM_RTLinker
> ld $3,0($2)           << WRONG
> dla $2,MM_RTLinker
> ld $2,0($2)           << WRONG
> daddu $4,$3,$2
> dla $3,MM_RTLinker
> ld $2,128($3)         << OK
> andi $2,$2,0x0
> or $2,$2,$4
> sd $2,128($3)         << OK
> .L65:
> .LBE9:
> .stabn 68,0,224,.LM121
> .LM121:
> move $sp,$fp
> ld $31,56($sp)
> ld $fp,48($sp)
> ld $28,40($sp)
> daddiu $sp,$sp,64
> j $31
> .end RTLinker__ExpandModuleTable
>
> Here is corrected code:
>
> RTLinker__ExpandModuleTable:
> .frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0
> .mask 0xd0000000,-8
> .fmask 0x00000000,0
> daddiu $sp,$sp,-64
> sd $31,56($sp)
> sd $fp,48($sp)
> sd $28,40($sp)
> move $fp,$sp
> lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
> daddu $28,$28,$25
> daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
> .LBB9:
> .stabn 68,0,208,.LM109
> .LM109:
> sd $0,16($fp)
> .stabn 68,0,210,.LM110
> .LM110:
> dla $2,MM_RTLinker
> daddiu $2,$2,136     << CORRECT (offset seems low, but at least it  
> isn't zero)
> ld $2,0($2)
> bne $2,$0,.L61
> .stabn 68,0,212,.LM111
> .LM111:
> dla $2,MM_RTLinker
> daddiu $3,$2,128     << CORRECT
> li $2,500   # 0x1f4
> sd $2,0($3)
> .stabn 68,0,213,.LM112
> .LM112:
> li $4,4000   # 0xfa0
> jal malloc
> sd $2,0($fp)
> dla $2,MM_RTLinker
> daddiu $3,$2,136      << CORRECT
> ld $2,0($fp)
> sd $2,0($3)
> .stabn 68,0,214,.LM113
> .LM113:
> dla $2,MM_RTLinker
> daddiu $2,$2,136      << CORRECT
> ld $2,0($2)
> bne $2,$0,.L65
> jal abort
> b .L65
> .L61:
>
> I still dump core with this change, but I get much further.
> I have not tested this change at all otherwise -- i.e. on working  
> platforms.
>
> I have not debugged through the gcc/m3cg code to see why
> the offsets are only sometimes lost, or why other platforms
> have no problem. Is MIPS's "global pointer" unusual these days?
> I know SPARC has similar, and all the non-x86, non-AMD64 NT platforms
> had similar, but that's essentially nothing now (I'm not counting  
> IA64).
> I know Linux has PLT (procedure linkage table) and GOT (global  
> offset table).
> Same thing?
> I haven't looked much at other MIPS ports yet.
>
> http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8
>
> is what made it look about like it does -- in particularing using  
> BIT_FIELD_REF.
>
> 1.7:
>
> static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */
>     tree v;
>     int o;
>     tree src_t, dest_t;
>     m3_type src_T, dest_T;
> {
>  if (o == 0 && TREE_TYPE (v) == src_t) {
>    EXPR_PUSH (v);
>  } else {
>    tree adr = m3_build1 (ADDR_EXPR, t_addr, v);
>    if (o != 0) {
>      adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o /  
> BITS_PER_UNIT));
>    }
>    EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t,
>                          m3_cast (build_pointer_type (src_t), adr)));
>  }
> #if 1
>  if (src_T != dest_T) {
>    EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1));
>  }
>  if (debug_vars) {
>    const char *name = "noname";
>    if (v != 0 && DECL_NAME(v) != 0) {
>      name = IDENTIFIER_POINTER(DECL_NAME(v));
>    }
>    fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n",  
> name,
>            o, src_T, dest_T);
>  }
> #else
>  if (src_T != dest_T) {
>    if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) {
>      EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1));
>    } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) {
>      EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1));
>    } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) {
>      /* okay */
>    } else {
>      fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n",
>                  src_T, dest_T);
>    }
>  }
> #endif
> }
> static void
> m3_store (v, o, src_t, dest_t)  /* GCC32OK */
>     tree v;
>     int o;
>     tree src_t, dest_t;
> {
>  tree lhs, rhs;
>  if (TREE_TYPE (EXPR_REF (-1)) == src_t) {
>    rhs = EXPR_REF (-1);
>  } else {
>    rhs = m3_cast (src_t, EXPR_REF (-1));
>  }
>  if (o == 0 && TREE_TYPE (v) == dest_t) {
>    lhs = v;
>  } else {
>    tree f = make_node (FIELD_DECL);
>    TREE_TYPE (f) = dest_t;
>    DECL_ALIGN (f) = TYPE_ALIGN (dest_t);
>    DECL_SIZE (f) = TYPE_SIZE (dest_t);
>    DECL_MODE (f) = TYPE_MODE (dest_t);
>    DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT);
>    DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT);
>    DECL_FIELD_CONTEXT (f) = TREE_TYPE (v);
>    lhs = m3_build2 (COMPONENT_REF, dest_t, v, f);
>  }
> #if 1
>    expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0,  
> 0);
> #else
>  if (dest_t ==  NULL) {
>    expand_assignment (lhs, rhs, 0, 0);
>  } else {
>    expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0,  
> 0);
>  }
> #endif
>  EXPR_POP ();
> }
>
> 1.8:
> static void
> m3_load (tree v, int o,
>  tree src_t, m3_type src_T,
>  tree dst_t, m3_type dst_T) /* GCC32OK */
> {
>  if (o == 0 && TREE_TYPE (v) == src_t) {
>    EXPR_PUSH (v);
>  } else {
>    EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),
>     bitsize_int (o)));
>  }
>  if (src_T != dst_T) {
>    EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1));
>  }
>  if (option_vars_trace) {
>    const char *name = "noname";
>    if (v != 0 && DECL_NAME(v) != 0) {
>      name = IDENTIFIER_POINTER(DECL_NAME(v));
>    }
>    fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n",  
> name,
>     o, src_T, dst_T);
>  }
> }
> static void
> m3_store (tree v, int o, tree src_t, tree dst_t)  /* GCC32OK */
> {
>  tree lhs, rhs;
>  if (TREE_TYPE (EXPR_REF (-1)) == src_t) {
>    rhs = EXPR_REF (-1);
>  } else {
>    rhs = m3_cast (src_t, EXPR_REF (-1));
>  }
>  if (o == 0 && TREE_TYPE (v) == dst_t) {
>    lhs = v;
>  } else {
>    lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),
>       bitsize_int (o));
>  }
>  if (src_t != dst_t) {
>    rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs);
>  }
>  expand_assignment (lhs, rhs, 0, 0);
>  EXPR_POP ();
> }
>
> - Jay
>
> <AMD64_LINUX_RTLinker.mc><MIPS64_OPENBSD_RTLinker.mc>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081112/6950a8b4/attachment.html>

From jay.krell at cornell.edu  Thu Nov 13 01:04:32 2008
From: jay.krell at cornell.edu (Jay)
Date: Thu, 13 Nov 2008 00:04:32 +0000
Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
In-Reply-To: <80AE04BD-4819-4085-8EBE-5996E8E6DB93@cs.purdue.edu>
References: <COL101-W3DB0BA622C4D31A82B602E6180@phx.gbl>
	<C47F6B5A-F154-4485-B839-96185B59E3B6@cs.purdue.edu>
	<COL101-W2CFF8666D1486BD514159E61B0@phx.gbl>
	<B9A95C1E-B873-40A9-A045-CD38E32CA5D0@cs.purdue.edu>
	<BAY116-W5161CD049D8CA0E7ABAE4AE6140@phx.gbl>
	<80AE04BD-4819-4085-8EBE-5996E8E6DB93@cs.purdue.edu>
Message-ID: <COL101-W60267BCFBF723F2491E3CEE6170@phx.gbl>


 > Jay, do you have access to an AMD64_DARWIN box?
 
No, sorry, I don't yet and not sure when I will.
Unless it is easy enough to run on non-Apple hardware.
I'll have to debug it (m3cg) then.
 
 - Jay



CC: m3devel at elegosoft.comFrom: hosking at cs.purdue.eduTo: jay.krell at cornell.eduSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Wed, 12 Nov 2008 20:46:53 +0000Jay, do you have access to an AMD64_DARWIN box?   I did get things working on that 64-bit platform.   Or is it an endianness issue you are dealing with.  In any case, I will not have a lot of bandwidth for doing much in the way of tackling this unless it becomes critical path for me soon.




Antony Hosking | Associate Professor | Computer Science | Purdue University
305 N. University Street | West Lafayette | IN 47907 | USA
Office +1 765 494 6001 | Mobile +1 765 427 5484


On 12 Nov 2008, at 07:57, Jay wrote:

Target.Little_endian is used to drive some target-specificity in the IR.   I guess, after all, that is why you have to specify it. The IR for RTLinker.m3 is identical for MIPS64_OPENBSD and SPARC64_SOLARIS, once I set them both to unaligned_procedures, which they both need. (Tangent, I ran m3cgcat on the .ms file a few times instead of the .mc file; that makes it crash.) I can't claim SPARC64_SOLARIS is any paragon of working and correct though, by far.I should also be able to compare with SPARC64_OPENBSD, I forgot about that, and didn't have an m3cg.I only ran SPARC64_SOLARIS far enough to discover it needed unaligned procedures.I think I should fork linux-common and solaris-common off of the minimal 29k openbsd-common, rather than worry about all the stuff in solaris-2-x and linux-libc, over 100k each. Then SPARC64_SOLARIS could/would be in better shape probably. MIPS64_OPENBSD cm3 gets further with unaligned_procedures, along with the parse.c fix -- it gets to a bus error in RunMainBody.Again an alignment problem, though I think maybe memory memory trashing.The fault occurs here:     desc_offset := init_depth * ADRSIZE (InitDesc);      desc := init_stack + desc_offset;      init_depth := desc.low_link; <<   InitDesc = RECORD    module   : RT0.ModulePtr;    low_link : INTEGER;  END;desc_offset appears to not be a multiple of 4. I will look into that. Long story short -- consider again the parse.c change?I admit I still haven't debugged it (or rather, the code that consumes the data produced by it).  - Jay

From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduCC: m3devel at elegosoft.comSubject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Sun, 9 Nov 2008 11:10:54 +0000I agree. I am suspicious too. I will investigate more. Thanks.  - Jay

From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 9 Nov 2008 10:01:58 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
Aren't the integer values supposed to be written endian-independently in the IR?  I am suspicious about these differences.




On 9 Nov 2008, at 01:25, Jay wrote:

Thanks for the tip/direction. Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the same" but for two systematic diffs:  - AMD64_LINUX has a bunch of added calls to check_nil.   This is a "bug" in it, inoptimal but ok code, because "first readable address" is 0. It should probably be 4k. I should write a program that iterates starting at 0 to find the value.  (or better yet, cm3 should have a -configure option that does this? -- for configuration values for which there is a safe default; in general, imho, the targets should be auto configured via C, assuming there is a native development environment available).  - some integers are different (again, systematically)  AMD64_LINUX: init_int  656 13 Int.64 init_chars  664 "    typecode " init_int  680 2 Int.64 init_var  688 v.1 0 init_int  696 13 Int.64 init_chars  704 "    typeid   " init_int  720 2 Int.64 init_var  728 v.1 0 init_int  736 13 Int.64 init_chars  744 "RunMainBody: " init_int  760 2 Int.64 init_var  768 v.1 0 init_int  776 19 Int.64 init_chars  784 "RunMainBody: exec: " init_int  808 2 Int.64 init_var  816 v.1 0 init_int  824 17 Int.64 init_chars  832 "ExpandInitStack: "MIPS64_OPENBSD:  init_chars  664 "    typecode " init_int  680 8796093022208 Int.64 init_var  688 v.1 0 init_int  696 13 Int.64 init_chars  704 "    typeid   " init_int  720 8796093022208 Int.64 init_var  728 v.1 0 init_int  736 13 Int.64 init_chars  744 "RunMainBody: " init_int  760 8796093022208 Int.64 init_var  768 v.1 0 init_int  776 19 Int.64 init_chars  784 "RunMainBody: exec: " init_int  808 8796093022208 Int.64 init_var  816 v.1 0This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where little endian has 2. I don't know what these numbers represent.I should look at other systems -- preferably a working 64 bit big endian system, if there are any. Could be flags/type with text literals? (Btw, I always leave the TextLiteral.i3 change on my system, lowering the maximum literal length to around 4gig even on 64 bit targets, so 32 bit can bootstrap 64 bit...need to fix the compiler here to do target arithmetic instead of host arithmetic...) Thanks, - Jay

From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sat, 8 Nov 2008 22:55:54 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Have you checked to make sure that the pre-gcc IR makes sense?  Does it see these values as 32-bit or 64-bit?  The reason I ask is that there may be something wrong in your m3middle target specs that cause bad IR.  Once we know the IR looks right then we can try and fix the problem.



Antony Hosking | Associate Professor | Computer Science | Purdue University
305 N. University Street | West Lafayette | IN 47907 | USA
Office +1 765 494 6001 | Mobile +1 765 427 5484


On 8 Nov 2008, at 15:45, Jay wrote:

truncated as usual...along with losing a lot of whitespace..Let's see...Tony, how about switching these #if's?I explain some of why below.static voidm3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){ tree val;#if 0 if (o != 0 || TREE_TYPE (v) != dst_t) {   v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),    bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) {   v = m3_build1 (ADDR_EXPR, t_addr, v);   v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));   v = m3_build1 (INDIRECT_REF, dst_t,    m3_cast (build_pointer_type (dst_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) {   val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP ();}m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){#if 0 if (o != 0 || TREE_TYPE (v) != src_t) {   v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),    bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) {   v = m3_build1 (ADDR_EXPR, t_addr, v);   v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));   v = m3_build1 (INDIRECT_REF, src_t,    m3_cast (build_pointer_type (src_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) {   v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v);}I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).In an early version it seems that many loads/stores of global variables is wrong.The "first" code affected is RTLinker_ExpandModuleTable.PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS;  n_bytes: INTEGER; BEGIN   IF (modules = NIL) THEN     (* first time... *)     max_modules := InitialTableSize;     modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr));     IF (modules = NIL) THEN Cstdlib.abort (); END;   ELSE     n_bytes := max_modules * BYTESIZE (RT0.ModulePtr);     new_mods := Cstdlib.malloc (n_bytes + n_bytes);     IF (new_mods = NIL) THEN Cstdlib.abort (); END;     EVAL Cstring.memcpy (new_mods, modules, n_bytes);     Cstdlib.free (modules);     modules := new_mods;     INC (max_modules, max_modules);   END; END ExpandModuleTable;It should read back modules as NIL.What happens is it reads back modules as not NIL,and then max_modules should also be zero, but is "large", leading to malloc to fail,and abort called.The output of cm3cg -y is reasonable.It indicates offset loads.What is incorrect is that in many loads and stores, but not all, theoffsets are lost.(923) begin_procedure procedure RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable(924) set_source_line source line  208(925) load_nil(926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb(927) set_source_line source line  207(928) set_source_line source line  210(929) load_nil(930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xbHere is the incorrect code:.globl RTLinker__ExpandModuleTable.stabd 46,0,0.stabn 68,0,206,.LM108.LM108:.LFBB8:.set nomips16.ent RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable:.frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerld $2,0($2)      << WRONGbne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $3,MM_RTLinkerld $2,128($3)andi $2,$2,0x0ori $2,$2,0x1f4sd $2,128($3).stabn 68,0,213,.LM112.LM112:li $4,4000   # 0xfa0jal mallocsd $2,0($fp)ld $4,0($fp)dla $3,MM_RTLinkerld $2,136($3)andi $2,$2,0x0or $2,$4,$2sd $2,136($3)     << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerld $2,0($2)       << WRONG bne $2,$0,.L65jal abortb .L65.L61:.stabn 68,0,216,.LM114.LM114:dla $2,MM_RTLinkerld $2,0($2)        << WRONG dsll $2,$2,3sd $2,8($fp).stabn 68,0,217,.LM115.LM115:ld $3,8($fp)ld $2,8($fp)daddu $2,$3,$2move $4,$2jal mallocsd $2,0($fp)ld $2,0($fp)sd $2,16($fp).stabn 68,0,218,.LM116.LM116:ld $2,16($fp)bne $2,$0,.L64jal abort.L64:.stabn 68,0,219,.LM117.LM117:ld $4,16($fp)dla $2,MM_RTLinkerld $2,0($2)          << WRONGld $3,8($fp)move $5,$2move $6,$3jal memcpysd $2,0($fp).stabn 68,0,220,.LM118.LM118:dla $2,MM_RTLinkerld $2,0($2)           << WRONGmove $4,$2jal free.stabn 68,0,221,.LM119.LM119:ld $4,16($fp)dla $3,MM_RTLinkerld $2,136($3)         << OKandi $2,$2,0x0or $2,$4,$2sd $2,136($3)         << OK .stabn 68,0,222,.LM120.LM120:dla $2,MM_RTLinkerld $3,0($2)           << WRONGdla $2,MM_RTLinkerld $2,0($2)           << WRONGdaddu $4,$3,$2dla $3,MM_RTLinkerld $2,128($3)         << OKandi $2,$2,0x0or $2,$2,$4sd $2,128($3)         << OK.L65:.LBE9:.stabn 68,0,224,.LM121.LM121:move $sp,$fpld $31,56($sp)ld $fp,48($sp)ld $28,40($sp)daddiu $sp,$sp,64j $31.end RTLinker__ExpandModuleTableHere is corrected code:RTLinker__ExpandModuleTable:.frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerdaddiu $2,$2,136     << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2)bne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $2,MM_RTLinkerdaddiu $3,$2,128     << CORRECT li $2,500   # 0x1f4sd $2,0($3).stabn 68,0,213,.LM112.LM112:li $4,4000   # 0xfa0jal mallocsd $2,0($fp)dla $2,MM_RTLinkerdaddiu $3,$2,136      << CORRECT ld $2,0($fp)sd $2,0($3).stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerdaddiu $2,$2,136      << CORRECT ld $2,0($2)bne $2,$0,.L65jal abortb .L65.L61:I still dump core with this change, but I get much further.I have not tested this change at all otherwise -- i.e. on working platforms.I have not debugged through the gcc/m3cg code to see whythe offsets are only sometimes lost, or why other platformshave no problem. Is MIPS's "global pointer" unusual these days?I know SPARC has similar, and all the non-x86, non-AMD64 NT platformshad similar, but that's essentially nothing now (I'm not counting IA64).I know Linux has PLT (procedure linkage table) and GOT (global offset table).Same thing?I haven't looked much at other MIPS ports yet.http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8is what made it look about like it does -- in particularing using BIT_FIELD_REF.1.7:static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */    tree v;    int o;    tree src_t, dest_t;    m3_type src_T, dest_T;{ if (o == 0 && TREE_TYPE (v) == src_t) {   EXPR_PUSH (v); } else {   tree adr = m3_build1 (ADDR_EXPR, t_addr, v);   if (o != 0) {     adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT));   }   EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t,                         m3_cast (build_pointer_type (src_t), adr))); }#if 1 if (src_T != dest_T) {   EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) {   const char *name = "noname";   if (v != 0 && DECL_NAME(v) != 0) {     name = IDENTIFIER_POINTER(DECL_NAME(v));   }   fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n", name,           o, src_T, dest_T); }#else if (src_T != dest_T) {   if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) {     EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1));   } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) {     EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1));   } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) {     /* okay */   } else {     fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n",                 src_T, dest_T);   } }#endif}static voidm3_store (v, o, src_t, dest_t)  /* GCC32OK */    tree v;    int o;    tree src_t, dest_t;{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) {   rhs = EXPR_REF (-1); } else {   rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) {   lhs = v; } else {   tree f = make_node (FIELD_DECL);   TREE_TYPE (f) = dest_t;   DECL_ALIGN (f) = TYPE_ALIGN (dest_t);   DECL_SIZE (f) = TYPE_SIZE (dest_t);   DECL_MODE (f) = TYPE_MODE (dest_t);   DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT);   DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT);   DECL_FIELD_CONTEXT (f) = TREE_TYPE (v);   lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); }#if 1   expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);#else if (dest_t ==  NULL) {   expand_assignment (lhs, rhs, 0, 0); } else {   expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); }#endif EXPR_POP ();}1.8:static voidm3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */{ if (o == 0 && TREE_TYPE (v) == src_t) {   EXPR_PUSH (v); } else {   EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),    bitsize_int (o))); } if (src_T != dst_T) {   EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) {   const char *name = "noname";   if (v != 0 && DECL_NAME(v) != 0) {     name = IDENTIFIER_POINTER(DECL_NAME(v));   }   fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n", name,    o, src_T, dst_T); }}static voidm3_store (tree v, int o, tree src_t, tree dst_t)  /* GCC32OK */{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) {   rhs = EXPR_REF (-1); } else {   rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) {   lhs = v; } else {   lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),      bitsize_int (o)); } if (src_t != dst_t) {   rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP ();}- Jay<AMD64_LINUX_RTLinker.mc><MIPS64_OPENBSD_RTLinker.mc>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081113/2f8b22da/attachment.html>

From rodney.bates at wichita.edu  Thu Nov 13 16:45:34 2008
From: rodney.bates at wichita.edu (Rodney M. Bates)
Date: Thu, 13 Nov 2008 09:45:34 -0600
Subject: [M3devel] Still can't bootstrap cm3
In-Reply-To: <20081112081855.alr1p7k4ggw0k4k0@mail.elegosoft.com>
References: <4919ED6D.6000700@wichita.edu>
	<20081112081855.alr1p7k4ggw0k4k0@mail.elegosoft.com>
Message-ID: <491C4B9E.4010307@wichita.edu>

Olaf Wagner wrote:
> 
> it's not easy to understand what has gone wrong at this point.
> Usually, if you have a working system, cm3/scripts/upgrade.sh
> should be able to bootstrap to a new set of sources. This is tested
> every day in the tinderbox release tests, too. As the last builds seem
> to have succeeded, I don't think there is a real problem in our
> code base currently. You may want to have a look at test_build_current
> in cm3/scripts/regression/defs.sh to see what's done in the regression
> builds.
> 
> I'd suggest you either start again from the last release or
> one of the latest snapshots available from the build machine.

Yes, that's what I always end up doing.  But it would be nice to be
able someday to get my local compiler to rebuild itself :-(.

But even that is not working for me.  Reinstalling a snapshot from
2008-6-6 and a fresh cvs checkout, upgrade.sh gives the missing
.M3EXPORTS failure, and do-cm3-core.sh (after realclean) gets bad
version stamps on Compiler.Platform and Compiler.ThisPlatform,
while compiling SocketPosix.m3.

Going to the most recent (at the time) snapshot of 2008-11-12,
do-cm3-core.sh will run without failures, but leaves a compiler
that always gets the undefined ROOT failure, even trying to compile
a trivial program.

> I assume you are not building a cross compiler, but just trying
> to perform an upgrade.

That's right.  The only reason I messed with the cross scripts was that
I hoped they might take care of the necessary bootstrapping in a way
the would work.

> 
> Daily snapshots are available from
> http://www.opencm3.net/snaps/snapshot-index.html
> 
> I hope this helps,
> 
> Olaf
> 
>


From jay.krell at cornell.edu  Thu Nov 13 17:37:47 2008
From: jay.krell at cornell.edu (Jay)
Date: Thu, 13 Nov 2008 16:37:47 +0000
Subject: [M3devel] Still can't bootstrap cm3
In-Reply-To: <491C4B9E.4010307@wichita.edu>
References: <4919ED6D.6000700@wichita.edu>
	<20081112081855.alr1p7k4ggw0k4k0@mail.elegosoft.com> 
	<491C4B9E.4010307@wichita.edu>
Message-ID: <BAY118-W1318AE445EC7D9ACF911E8E6170@phx.gbl>


Randy, ROOT not being defined is a function of your config files.
Look at mine. Mine are willing to ferry it from an environment variable.
The various scripts figure it out and set it on the command line.
 e.g. sysinfo.sh
 
Granted..I don't think ROOT is supposed to be required.
ROOT is the root of the source tree, as you correctly?successfully worked around.
You are supposed to be able to use the compiler without a source tree.
But you can't, um, build the source tree without a source tree.
 
I am very tempted to have cm3.exe set ROOT itself,
but it is a bit inappropriate actually.
 
PERHAPS if it was renamed CM3_ROOT...
 
Olaf isn't saying "just use a current release", he is
saying "start with a current release" and then try to build locally.
 
You cannot necessarily bootstrap from arbitrarily old releases.
But you should definitely be able to bootstrap from current snapshots.
 
It does work.
 
Whenever you get into a "bad state" -- after any failed run,
I encourage you to blow away the entire cm3 install and "start over".
 
Several "partial builds" will leave you with a "mixed state".
 
Some of the scripts do or used to "backup" the pkg store,
but I don't think it was worth the effort...the effort I put
in to clone the .sh in .cmd. I think I punted doing that in the *.py,
and I don't miss it.
 
I also recommend trying the *.py files.
 
> that always gets the undefined ROOT failure, even trying to compile> a trivial program.
Hm. Could be your config file wants it?
Could be my config files wants it?
I guess the usual refrain -- send us your config file(s).
I can dig more later..maybe days..busy with other stuff..
 
 
 - Jay> Date: Thu, 13 Nov 2008 09:45:34 -0600> From: rodney> To: m3devel> Subject: Re: [M3devel] Still can't bootstrap cm3> > Olaf wrote:> > > > it's not easy to understand what has gone wrong at this point.> > Usually, if you have a working system, cm3/scripts/upgrade.sh> > should be able to bootstrap to a new set of sources. This is tested> > every day in the tinderbox release tests, too. As the last builds seem> > to have succeeded, I don't think there is a real problem in our> > code base currently. You may want to have a look at test_build_current> > in cm3/scripts/regression/defs.sh to see what's done in the regression> > builds.> > > > I'd suggest you either start again from the last release or> > one of the latest snapshots available from the build machine.> > Yes, that's what I always end up doing. But it would be nice to be> able someday to get my local compiler to rebuild itself :-(.> > But even that is not working for me. Reinstalling a snapshot from> 2008-6-6 and a fresh cvs checkout, upgrade.sh gives the missing> .M3EXPORTS failure, and do-cm3-core.sh (after realclean) gets bad> version stamps on Compiler.Platform and Compiler.ThisPlatform,> while compiling SocketPosix.m3.> > Going to the most recent (at the time) snapshot of 2008-11-12,> do-cm3-core.sh will run without failures, but leaves a compiler> that always gets the undefined ROOT failure, even trying to compile> a trivial program.> > > I assume you are not building a cross compiler, but just trying> > to perform an upgrade.> > That's right. The only reason I messed with the cross scripts was that> I hoped they might take care of the necessary bootstrapping in a way> the would work.> > > > > Daily snapshots are available from> > http://www.opencm3.net/snaps/snapshot-index.html> > > > I hope this helps,> > > > Olaf> > > >
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081113/3998c75f/attachment.html>

From jay.krell at cornell.edu  Thu Nov 13 17:54:55 2008
From: jay.krell at cornell.edu (Jay)
Date: Thu, 13 Nov 2008 16:54:55 +0000
Subject: [M3devel] Still can't bootstrap cm3
In-Reply-To: <BAY118-W1318AE445EC7D9ACF911E8E6170@phx.gbl>
References: <4919ED6D.6000700@wichita.edu>
	<20081112081855.alr1p7k4ggw0k4k0@mail.elegosoft.com> 
	<491C4B9E.4010307@wichita.edu> 
	<BAY118-W1318AE445EC7D9ACF911E8E6170@phx.gbl>
Message-ID: <BAY117-W27AE620521647E466EC8D2E6170@phx.gbl>


Also, until you find something that works:
 export OMIT_GCC=yes
 
Otherwise you are going to waste a lot more time experimenting.
 
 - Jay



From: jay.krell at cornell.eduTo: rodney.bates at wichita.edu; m3devel at elegosoft.comDate: Thu, 13 Nov 2008 16:37:47 +0000Subject: Re: [M3devel] Still can't bootstrap cm3

Randy, ROOT not being defined is a function of your config files.Look at mine. Mine are willing to ferry it from an environment variable.The various scripts figure it out and set it on the command line. e.g. sysinfo.sh Granted..I don't think ROOT is supposed to be required.ROOT is the root of the source tree, as you correctly?successfully worked around.You are supposed to be able to use the compiler without a source tree.But you can't, um, build the source tree without a source tree. I am very tempted to have cm3.exe set ROOT itself,but it is a bit inappropriate actually. PERHAPS if it was renamed CM3_ROOT... Olaf isn't saying "just use a current release", he issaying "start with a current release" and then try to build locally. You cannot necessarily bootstrap from arbitrarily old releases.But you should definitely be able to bootstrap from current snapshots. It does work. Whenever you get into a "bad state" -- after any failed run,I encourage you to blow away the entire cm3 install and "start over". Several "partial builds" will leave you with a "mixed state". Some of the scripts do or used to "backup" the pkg store,but I don't think it was worth the effort...the effort I putin to clone the .sh in .cmd. I think I punted doing that in the *.py,and I don't miss it. I also recommend trying the *.py files. > that always gets the undefined ROOT failure, even trying to compile> a trivial program.Hm. Could be your config file wants it?Could be my config files wants it?I guess the usual refrain -- send us your config file(s).I can dig more later..maybe days..busy with other stuff..   - Jay> Date: Thu, 13 Nov 2008 09:45:34 -0600> From: rodney> To: m3devel> Subject: Re: [M3devel] Still can't bootstrap cm3> > Olaf wrote:> > > > it's not easy to understand what has gone wrong at this point.> > Usually, if you have a working system, cm3/scripts/upgrade.sh> > should be able to bootstrap to a new set of sources. This is tested> > every day in the tinderbox release tests, too. As the last builds seem> > to have succeeded, I don't think there is a real problem in our> > code base currently. You may want to have a look at test_build_current> > in cm3/scripts/regression/defs.sh to see what's done in the regression> > builds.> > > > I'd suggest you either start again from the last release or> > one of the latest snapshots available from the build machine.> > Yes, that's what I always end up doing. But it would be nice to be> able someday to get my local compiler to rebuild itself :-(.> > But even that is not working for me. Reinstalling a snapshot from> 2008-6-6 and a fresh cvs checkout, upgrade.sh gives the missing> .M3EXPORTS failure, and do-cm3-core.sh (after realclean) gets bad> version stamps on Compiler.Platform and Compiler.ThisPlatform,> while compiling SocketPosix.m3.> > Going to the most recent (at the time) snapshot of 2008-11-12,> do-cm3-core.sh will run without failures, but leaves a compiler> that always gets the undefined ROOT failure, even trying to compile> a trivial program.> > > I assume you are not building a cross compiler, but just trying> > to perform an upgrade.> > That's right. The only reason I messed with the cross scripts was that> I hoped they might take care of the necessary bootstrapping in a way> the would work.> > > > > Daily snapshots are available from> > http://www.opencm3.net/snaps/snapshot-index.html> > > > I hope this helps,> > > > Olaf> > > >
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081113/00c9851a/attachment.html>

From hosking at cs.purdue.edu  Thu Nov 13 18:06:22 2008
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Thu, 13 Nov 2008 11:06:22 -0600
Subject: [M3devel] Still can't bootstrap cm3
In-Reply-To: <491C4B9E.4010307@wichita.edu>
References: <4919ED6D.6000700@wichita.edu>
	<20081112081855.alr1p7k4ggw0k4k0@mail.elegosoft.com>
	<491C4B9E.4010307@wichita.edu>
Message-ID: <77D02692-31FE-4326-8E0E-8758C323CDE9@cs.purdue.edu>

This is odd.  I've been rebuilding my local compiler by hand several  
times in the last few weeks.  I will try the scripts and follow up.

Antony Hosking | Associate Professor | Computer Science | Purdue  
University
305 N. University Street | West Lafayette | IN 47907 | USA
Office +1 765 494 6001 | Mobile +1 765 427 5484



On 13 Nov 2008, at 09:45, Rodney M. Bates wrote:

> Olaf Wagner wrote:
>> it's not easy to understand what has gone wrong at this point.
>> Usually, if you have a working system, cm3/scripts/upgrade.sh
>> should be able to bootstrap to a new set of sources. This is tested
>> every day in the tinderbox release tests, too. As the last builds  
>> seem
>> to have succeeded, I don't think there is a real problem in our
>> code base currently. You may want to have a look at  
>> test_build_current
>> in cm3/scripts/regression/defs.sh to see what's done in the  
>> regression
>> builds.
>> I'd suggest you either start again from the last release or
>> one of the latest snapshots available from the build machine.
>
> Yes, that's what I always end up doing.  But it would be nice to be
> able someday to get my local compiler to rebuild itself :-(.
>
> But even that is not working for me.  Reinstalling a snapshot from
> 2008-6-6 and a fresh cvs checkout, upgrade.sh gives the missing
> .M3EXPORTS failure, and do-cm3-core.sh (after realclean) gets bad
> version stamps on Compiler.Platform and Compiler.ThisPlatform,
> while compiling SocketPosix.m3.
>
> Going to the most recent (at the time) snapshot of 2008-11-12,
> do-cm3-core.sh will run without failures, but leaves a compiler
> that always gets the undefined ROOT failure, even trying to compile
> a trivial program.
>
>> I assume you are not building a cross compiler, but just trying
>> to perform an upgrade.
>
> That's right.  The only reason I messed with the cross scripts was  
> that
> I hoped they might take care of the necessary bootstrapping in a way
> the would work.
>
>> Daily snapshots are available from
>> http://www.opencm3.net/snaps/snapshot-index.html
>> I hope this helps,
>> Olaf
>>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081113/d9f9e670/attachment.html>

From rodney.bates at wichita.edu  Fri Nov 14 00:29:17 2008
From: rodney.bates at wichita.edu (Rodney M. Bates)
Date: Thu, 13 Nov 2008 17:29:17 -0600
Subject: [M3devel] The undefined ROOT problem, rebuilding cm3
Message-ID: <491CB84D.4000008@wichita.edu>

Here is how the undefined ROOT problem happens when trying to rebuild
cm3 with itself.

I had been making the assumption that commands build, ship, clean, when
supplied to the do-cm3-*.sh scripts, did the same things as -build,
-ship, -clean supplied to the cm3 command.  Not so.

build actually does cm3 -build -override, as do buildlocal and empty
commands to the scripts. clean does cm3 -clean -override, as does cleanlocal.
There is a buildglobal command that does -build without -override, but
it also does a ship as well.  There appears to be no way to just build,
without either overriding or shipping.

So, I have been running do-cm3-*.sh with no argument (and sometimes build).
This starts a chain of events that eventually leads to an installed libm3
that always executes quake script bundle-ov.tmpl, and it requires that
either M3BUNDLE or ROOT be defined, neither of which normally is.  This
will happen trying to compile any program that uses libm3, which is just
about everything.

We should document this somewhere.


From jay.krell at cornell.edu  Fri Nov 14 02:44:44 2008
From: jay.krell at cornell.edu (Jay)
Date: Fri, 14 Nov 2008 01:44:44 +0000
Subject: [M3devel] The undefined ROOT problem, rebuilding cm3
In-Reply-To: <491CB84D.4000008@wichita.edu>
References: <491CB84D.4000008@wichita.edu>
Message-ID: <COL101-W79AA2376232A9B60AC3D33E6160@phx.gbl>


I don't know, but something doesn't seem right.
 
D:\cm3>dir /s/b bund* *ov* *temp* *tmp*...
D:\cm3\pkg\libm3\src\bundleintf\bundle.tmpl
 
if defined("M3BUNDLE")  _M3BUNDLE = M3BUNDLEelse  _M3BUNDLE = BIN_USE & SL & "m3bundle"end
D:\dev2\cm3.2>dir /s/b /a-d bund*tmp*D:\dev2\cm3.2\doc\help\gen_html\libm3\src\bundleintf\bundle.tmpl.htmlD:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle-ov.tmplD:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle.tmpl
 
:\dev2\cm3.2>dir /s/b /a-d bund*tmp*:\dev2\cm3.2\doc\help\gen_html\libm3\src\bundleintf\bundle.tmpl.html:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle-ov.tmpl:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle.tmpl
 
bundle-ov.tmp:
 
if defined("M3BUNDLE")  _M3BUNDLE = M3BUNDLEelse  _M3BUNDLE = format(ROOT & SL & "m3-tools" & SL & "m3bundle" & SL & "%s" &                     SL & "m3bundle", BUILD_DIR)  M3BUNDLE = _M3BUNDLEend
 => this file requires ROOT or M3BUNDLE like you say, but this isn't normally used.
 
D:\dev2\cm3.2>dir /s/b m3makefile | findstr /i bundleintfD:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\m3makefile
 
if defined("_M3BUNDLE_OVERRIDE")  template  ("bundle-ov")endtemplate  ("bundle")
You defined _M3BUNDLE_OVERRIDE somewhere along the way?
 
Try replacing the one file with the other..
 
 - Jay> Date: Thu, 13 Nov 2008 17:29:17 -0600> From: rodney.bates at wichita.edu> To: m3devel at elegosoft.com> Subject: [M3devel] The undefined ROOT problem, rebuilding cm3> > Here is how the undefined ROOT problem happens when trying to rebuild> cm3 with itself.> > I had been making the assumption that commands build, ship, clean, when> supplied to the do-cm3-*.sh scripts, did the same things as -build,> -ship, -clean supplied to the cm3 command. Not so.> > build actually does cm3 -build -override, as do buildlocal and empty> commands to the scripts. clean does cm3 -clean -override, as does cleanlocal.> There is a buildglobal command that does -build without -override, but> it also does a ship as well. There appears to be no way to just build,> without either overriding or shipping.> > So, I have been running do-cm3-*.sh with no argument (and sometimes build).> This starts a chain of events that eventually leads to an installed libm3> that always executes quake script bundle-ov.tmpl, and it requires that> either M3BUNDLE or ROOT be defined, neither of which normally is. This> will happen trying to compile any program that uses libm3, which is just> about everything.> > We should document this somewhere.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081114/3c45491f/attachment.html>

From rodney.bates at wichita.edu  Fri Nov 14 03:01:30 2008
From: rodney.bates at wichita.edu (Rodney M. Bates)
Date: Thu, 13 Nov 2008 20:01:30 -0600
Subject: [M3devel] The undefined ROOT problem, rebuilding cm3
In-Reply-To: <COL101-W79AA2376232A9B60AC3D33E6160@phx.gbl>
References: <491CB84D.4000008@wichita.edu>
	<COL101-W79AA2376232A9B60AC3D33E6160@phx.gbl>
Message-ID: <491CDBFA.7030500@wichita.edu>

Jay wrote:
> I don't know, but something doesn't seem right.
>  
> D:\cm3>dir /s/b bund* *ov* *temp* *tmp*
> ...
> D:\cm3\pkg\libm3\src\bundleintf\bundle.tmpl
>  
> if defined("M3BUNDLE")
>   _M3BUNDLE = M3BUNDLE
> else
>   _M3BUNDLE = BIN_USE & SL & "m3bundle"
> end
> 
> D:\dev2\cm3.2>dir /s/b /a-d bund*tmp*
> D:\dev2\cm3.2\doc\help\gen_html\libm3\src\bundleintf\bundle.tmpl.html
> D:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle-ov.tmpl
> D:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle.tmpl
>  
> :\dev2\cm3.2>dir /s/b /a-d bund*tmp*
> :\dev2\cm3.2\doc\help\gen_html\libm3\src\bundleintf\bundle.tmpl.html
> :\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle-ov.tmpl
> :\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle.tmpl
>  
> bundle-ov.tmp:
>  
> if defined("M3BUNDLE")
>   _M3BUNDLE = M3BUNDLE
> else
>   _M3BUNDLE = format(ROOT & SL & "m3-tools" & SL & "m3bundle" & SL & "%s" &
>                      SL & "m3bundle", BUILD_DIR)
>   M3BUNDLE = _M3BUNDLE
> end
> 
>  => this file requires ROOT or M3BUNDLE like you say, but this isn't 
> normally used.
>  
> D:\dev2\cm3.2>dir /s/b m3makefile | findstr /i bundleintf
> D:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\m3makefile
>  
> if defined("_M3BUNDLE_OVERRIDE")
>   template  ("bundle-ov")
> end
> template  ("bundle")
> 
> You defined _M3BUNDLE_OVERRIDE somewhere along the way?

Maybe I should have given more detail on the chain of events.
You got most of it.  The missing piece is:

m3-libs/libm3/src/m3overrides contains:
_M3BUNDLE_OVERRIDE = "T"

So, 'do-cm3-*.sh build' runs 'cm3 -build -override', causing the
above file of quake code to be interpreted, defining _M3BUNDLE_OVERRIDE,
and the rest is as you have noted.

>  
> Try replacing the one file with the other..
>  
>  - Jay
> 
> 
> 
>  > Date: Thu, 13 Nov 2008 17:29:17 -0600
>  > From: rodney.bates at wichita.edu
>  > To: m3devel at elegosoft.com
>  > Subject: [M3devel] The undefined ROOT problem, rebuilding cm3
>  >
>  > Here is how the undefined ROOT problem happens when trying to rebuild
>  > cm3 with itself.
>  >
>  > I had been making the assumption that commands build, ship, clean, when
>  > supplied to the do-cm3-*.sh scripts, did the same things as -build,
>  > -ship, -clean supplied to the cm3 command. Not so.
>  >
>  > build actually does cm3 -build -override, as do buildlocal and empty
>  > commands to the scripts. clean does cm3 -clean -override, as does 
> cleanlocal.
>  > There is a buildglobal command that does -build without -override, but
>  > it also does a ship as well. There appears to be no way to just build,
>  > without either overriding or shipping.
>  >
>  > So, I have been running do-cm3-*.sh with no argument (and sometimes 
> build).
>  > This starts a chain of events that eventually leads to an installed libm3
>  > that always executes quake script bundle-ov.tmpl, and it requires that
>  > either M3BUNDLE or ROOT be defined, neither of which normally is. This
>  > will happen trying to compile any program that uses libm3, which is just
>  > about everything.
>  >
>  > We should document this somewhere.
> 



From hosking at cs.purdue.edu  Fri Nov 14 03:50:07 2008
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Thu, 13 Nov 2008 20:50:07 -0600
Subject: [M3devel] The undefined ROOT problem, rebuilding cm3
In-Reply-To: <491CDBFA.7030500@wichita.edu>
References: <491CB84D.4000008@wichita.edu>
	<COL101-W79AA2376232A9B60AC3D33E6160@phx.gbl>
	<491CDBFA.7030500@wichita.edu>
Message-ID: <74C195D7-3B89-4696-AB57-DE41DDD09E6D@cs.purdue.edu>

I generally run with buildship for these commands.

On 13 Nov 2008, at 20:01, Rodney M. Bates wrote:

> Jay wrote:
>> I don't know, but something doesn't seem right.
>> D:\cm3>dir /s/b bund* *ov* *temp* *tmp*
>> ...
>> D:\cm3\pkg\libm3\src\bundleintf\bundle.tmpl
>> if defined("M3BUNDLE")
>>  _M3BUNDLE = M3BUNDLE
>> else
>>  _M3BUNDLE = BIN_USE & SL & "m3bundle"
>> end
>> D:\dev2\cm3.2>dir /s/b /a-d bund*tmp*
>> D:\dev2\cm3.2\doc\help\gen_html\libm3\src\bundleintf\bundle.tmpl.html
>> D:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle-ov.tmpl
>> D:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle.tmpl
>> :\dev2\cm3.2>dir /s/b /a-d bund*tmp*
>> :\dev2\cm3.2\doc\help\gen_html\libm3\src\bundleintf\bundle.tmpl.html
>> :\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle-ov.tmpl
>> :\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle.tmpl
>> bundle-ov.tmp:
>> if defined("M3BUNDLE")
>>  _M3BUNDLE = M3BUNDLE
>> else
>>  _M3BUNDLE = format(ROOT & SL & "m3-tools" & SL & "m3bundle" & SL &  
>> "%s" &
>>                     SL & "m3bundle", BUILD_DIR)
>>  M3BUNDLE = _M3BUNDLE
>> end
>> => this file requires ROOT or M3BUNDLE like you say, but this isn't  
>> normally used.
>> D:\dev2\cm3.2>dir /s/b m3makefile | findstr /i bundleintf
>> D:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\m3makefile
>> if defined("_M3BUNDLE_OVERRIDE")
>>  template  ("bundle-ov")
>> end
>> template  ("bundle")
>> You defined _M3BUNDLE_OVERRIDE somewhere along the way?
>
> Maybe I should have given more detail on the chain of events.
> You got most of it.  The missing piece is:
>
> m3-libs/libm3/src/m3overrides contains:
> _M3BUNDLE_OVERRIDE = "T"
>
> So, 'do-cm3-*.sh build' runs 'cm3 -build -override', causing the
> above file of quake code to be interpreted, defining  
> _M3BUNDLE_OVERRIDE,
> and the rest is as you have noted.
>
>> Try replacing the one file with the other..
>>  - Jay
>> > Date: Thu, 13 Nov 2008 17:29:17 -0600
>> > From: rodney.bates at wichita.edu
>> > To: m3devel at elegosoft.com
>> > Subject: [M3devel] The undefined ROOT problem, rebuilding cm3
>> >
>> > Here is how the undefined ROOT problem happens when trying to  
>> rebuild
>> > cm3 with itself.
>> >
>> > I had been making the assumption that commands build, ship,  
>> clean, when
>> > supplied to the do-cm3-*.sh scripts, did the same things as -build,
>> > -ship, -clean supplied to the cm3 command. Not so.
>> >
>> > build actually does cm3 -build -override, as do buildlocal and  
>> empty
>> > commands to the scripts. clean does cm3 -clean -override, as does  
>> cleanlocal.
>> > There is a buildglobal command that does -build without - 
>> override, but
>> > it also does a ship as well. There appears to be no way to just  
>> build,
>> > without either overriding or shipping.
>> >
>> > So, I have been running do-cm3-*.sh with no argument (and  
>> sometimes build).
>> > This starts a chain of events that eventually leads to an  
>> installed libm3
>> > that always executes quake script bundle-ov.tmpl, and it requires  
>> that
>> > either M3BUNDLE or ROOT be defined, neither of which normally is.  
>> This
>> > will happen trying to compile any program that uses libm3, which  
>> is just
>> > about everything.
>> >
>> > We should document this somewhere.



From wagner at elegosoft.com  Fri Nov 14 09:35:07 2008
From: wagner at elegosoft.com (Olaf Wagner)
Date: Fri, 14 Nov 2008 09:35:07 +0100
Subject: [M3devel] The undefined ROOT problem, rebuilding cm3
In-Reply-To: <491CB84D.4000008@wichita.edu>
References: <491CB84D.4000008@wichita.edu>
Message-ID: <20081114093507.rp91pc17kg8cc8gw@mail.elegosoft.com>

Quoting "Rodney M. Bates" <rodney.bates at wichita.edu>:

> Here is how the undefined ROOT problem happens when trying to rebuild
> cm3 with itself.
>
> I had been making the assumption that commands build, ship, clean, when
> supplied to the do-cm3-*.sh scripts, did the same things as -build,
> -ship, -clean supplied to the cm3 command.  Not so.
>
> build actually does cm3 -build -override, as do buildlocal and empty
> commands to the scripts. clean does cm3 -clean -override, as does cleanlocal.
> There is a buildglobal command that does -build without -override, but
> it also does a ship as well.  There appears to be no way to just build,
> without either overriding or shipping.
>
> So, I have been running do-cm3-*.sh with no argument (and sometimes build).
> This starts a chain of events that eventually leads to an installed libm3
> that always executes quake script bundle-ov.tmpl, and it requires that
> either M3BUNDLE or ROOT be defined, neither of which normally is.  This
> will happen trying to compile any program that uses libm3, which is just
> about everything.
>
> We should document this somewhere.

Perhaps you could write up some paragraphs for the scripts' README
and the web pages?
I always seem to overlook or tend to obscure some crucial points for
other users :-/

Olaf
-- 
Olaf Wagner -- elego Software Solutions GmbH
                Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany
phone: +49 30 23 45 86 96  mobile: +49 177 2345 869  fax: +49 30 23 45 86 95
    http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin
Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194



From jay.krell at cornell.edu  Fri Nov 14 12:08:17 2008
From: jay.krell at cornell.edu (Jay)
Date: Fri, 14 Nov 2008 11:08:17 +0000
Subject: [M3devel] The undefined ROOT problem, rebuilding cm3
In-Reply-To: <74C195D7-3B89-4696-AB57-DE41DDD09E6D@cs.purdue.edu>
References: <491CB84D.4000008@wichita.edu>
	<COL101-W79AA2376232A9B60AC3D33E6160@phx.gbl>
	<491CDBFA.7030500@wichita.edu>
	<74C195D7-3B89-4696-AB57-DE41DDD09E6D@cs.purdue.edu>
Message-ID: <COL101-W32D8D3B50ACB900B343094E6160@phx.gbl>


> From: hosking>
> I generally run with buildship for these commands.
 
That, plus if you "buildlocal", you are supposed to be barred from "shipping".
That is, if you build with -override, you aren't supposed to be able to "ship".
"ship" is to copy from the source tree to the install tree.
I think I did see a problem once where it let me ship "local" stuff.
 
 - Jay
 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081114/c160ff55/attachment.html>

From rodney.bates at wichita.edu  Fri Nov 14 16:26:05 2008
From: rodney.bates at wichita.edu (Rodney M. Bates)
Date: Fri, 14 Nov 2008 09:26:05 -0600
Subject: [M3devel] The undefined ROOT problem, rebuilding cm3
In-Reply-To: <74C195D7-3B89-4696-AB57-DE41DDD09E6D@cs.purdue.edu>
References: <491CB84D.4000008@wichita.edu>
	<COL101-W79AA2376232A9B60AC3D33E6160@phx.gbl>
	<491CDBFA.7030500@wichita.edu>
	<74C195D7-3B89-4696-AB57-DE41DDD09E6D@cs.purdue.edu>
Message-ID: <491D988D.9080804@wichita.edu>

Tony Hosking wrote:
> I generally run with buildship for these commands.

And so will I, henceforth.

Rodney Bates




From jay.krell at cornell.edu  Fri Nov 14 21:24:55 2008
From: jay.krell at cornell.edu (Jay)
Date: Fri, 14 Nov 2008 20:24:55 +0000
Subject: [M3devel] Linux/amd64
Message-ID: <COL101-W20B494FEE8697CAC58E0DEE6160@phx.gbl>


I turned of gc just while running stubgen.
 
Leading to:
  http://m3.elegosoft.com/cm3/uploaded-archives/cm3-std-POSIX-AMD64_LINUX-d5.7.0.tar.gz  
  http://m3.elegosoft.com/cm3/uploaded-archives/cm3-min-POSIX-AMD64_LINUX-d5.7.0.tar.gz  

Please try them? See how well they work for you all?
I haven't run most of the stuff, but cm3 does build itself.
gc does usually?often work.

(saying "posix" I feel is redundant...)

 - Jay

From jay.krell at cornell.edu  Sun Nov 16 16:31:25 2008
From: jay.krell at cornell.edu (Jay)
Date: Sun, 16 Nov 2008 15:31:25 +0000
Subject: [M3devel] duplicated code?
Message-ID: <COL101-W82BC319326E633D918F12DE6100@phx.gbl>


Anyone want to clean up this duplicity?
 
 
D:\dev2\cm3.2>dir /s/b asttotype.m3  StubCode.m3D:\dev2\cm3.2\m3-comm\sharedobjgen\src\AstToType.m3D:\dev2\cm3.2\m3-comm\stubgen\src\AstToType.m3D:\dev2\cm3.2\m3-db\stablegen\src\AstToType.m3
D:\dev2\cm3.2\m3-comm\sharedobjgen\src\StubCode.m3D:\dev2\cm3.2\m3-comm\stubgen\src\StubCode.m3
 
somewhere in there, AMD64_LINUX tries to allocate a lot of memory, and fails
either there or soon thereafter.
The garbage collector is working.
 
 - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081116/deecc4a6/attachment.html>

From jay.krell at cornell.edu  Sun Nov 16 17:18:40 2008
From: jay.krell at cornell.edu (Jay)
Date: Sun, 16 Nov 2008 16:18:40 +0000
Subject: [M3devel] duplicated code?
In-Reply-To: <COL101-W82BC319326E633D918F12DE6100@phx.gbl>
References: <COL101-W82BC319326E633D918F12DE6100@phx.gbl>
Message-ID: <COL101-W16C880C91AE709A4E8463FE6100@phx.gbl>


Er, the large allocation does come from the gc itself.
 
Tony, is there is an assumption that the heap is contiguous?That calls to RTOS.GetMemory return adjacent addresses?
 
This code allocates a large array:
 
    IF desc = NIL OR newSideSpan # NUMBER(desc^) THEN      WITH newDesc = NEW(UNTRACED REF ARRAY OF Desc, newSideSpan) DO
I'll know more shortly.
 
I guess..it looks like the heap can be discontiguous, but
we do record keeping for what it all spans.
 
The comments say:
(* The array desc and the global variables p0, and p1 describe the pages   that are part of the traced heap.  Either p0 and p1 are equal to Nil and   no pages are allocated; or both are valid pages and page p is allocated   iff
|          p0 <= p < p1|      AND desc[p - p0] != Unallocated
 
Hm..
 
Grow (0x44000) => 0x2b1e45256000   total: 1.5M
GetUntracedOpenArray(0x1a80)   span: 6.6M   density: 24%stubgen: Processing RemoteView.TGetUntracedOpenArray(0x3f0)t1:0xct2:0xat3:0x1Grow (0x52000) => 0x2aaaaaaab000   total: 1.8M
GetUntracedOpenArray(0x1ce69fc8)
 
I have GetUntracedOpenArray printing how many bytes it is asked for.
t1,t2,t3 are just the lengths of the strings being concatented.
Grow(x)=>y means Grow allocated x bytes at address y.
 
So now, these two addresses 0x2aaaaaaab000 and 0x2b1e45256000  are very far apart, like 400gig.
And it seems the heap allocator wants to allocate an array to describe the pages.
 
Hm. Page size is no longer tied to the underlying system -- no longer vm-tied gc.
Perhaps perhaps blowing it up, to say, 1meg, will address this?
 
But really, an array to describe pages spanning the results of separate memory allocations, seems wrong.
A sparser data structure would be good, that could describe arbitrary sized runs of pages as being in the same state.
 
 - Jay



From: jay.krell at cornell.eduTo: m3devel at elegosoft.comDate: Sun, 16 Nov 2008 15:31:25 +0000Subject: [M3devel] duplicated code?

Anyone want to clean up this duplicity?  D:\dev2\cm3.2>dir /s/b asttotype.m3  StubCode.m3D:\dev2\cm3.2\m3-comm\sharedobjgen\src\AstToType.m3D:\dev2\cm3.2\m3-comm\stubgen\src\AstToType.m3D:\dev2\cm3.2\m3-db\stablegen\src\AstToType.m3D:\dev2\cm3.2\m3-comm\sharedobjgen\src\StubCode.m3D:\dev2\cm3.2\m3-comm\stubgen\src\StubCode.m3 somewhere in there, AMD64_LINUX tries to allocate a lot of memory, and failseither there or soon thereafter.The garbage collector is working.  - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081116/a28dfe37/attachment.html>

From hosking at cs.purdue.edu  Mon Nov 17 03:14:55 2008
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Sun, 16 Nov 2008 20:14:55 -0600
Subject: [M3devel] duplicated code?
In-Reply-To: <COL101-W16C880C91AE709A4E8463FE6100@phx.gbl>
References: <COL101-W82BC319326E633D918F12DE6100@phx.gbl>
	<COL101-W16C880C91AE709A4E8463FE6100@phx.gbl>
Message-ID: <4B6BDC53-DB07-4498-A0D4-CB383DFFD487@cs.purdue.edu>

On 16 Nov 2008, at 10:18, Jay wrote:

> Er, the large allocation does come from the gc itself.

What's being allocated?  The side array should not be too huge.

> Tony, is there is an assumption that the heap is contiguous?
> That calls to RTOS.GetMemory return adjacent addresses?

No assumption.

> This code allocates a large array:
>
>     IF desc = NIL OR newSideSpan # NUMBER(desc^) THEN
>       WITH newDesc = NEW(UNTRACED REF ARRAY OF Desc, newSideSpan) DO
>
> I'll know more shortly.
>
> I guess..it looks like the heap can be discontiguous, but
> we do record keeping for what it all spans.

Correct.

> The comments say:
> (* The array desc and the global variables p0, and p1 describe the  
> pages
>    that are part of the traced heap.  Either p0 and p1 are equal to  
> Nil and
>    no pages are allocated; or both are valid pages and page p is  
> allocated
>    iff
> |          p0 <= p < p1
> |      AND desc[p - p0] != Unallocated
>
>
> Hm..
>
> Grow (0x44000) => 0x2b1e45256000   total: 1.5M
> GetUntracedOpenArray(0x1a80)
>    span: 6.6M   density: 24%
> stubgen: Processing RemoteView.T
> GetUntracedOpenArray(0x3f0)
> t1:0xc
> t2:0xa
> t3:0x1
> Grow (0x52000) => 0x2aaaaaaab000   total: 1.8M
> GetUntracedOpenArray(0x1ce69fc8)
>
> I have GetUntracedOpenArray printing how many bytes it is asked for.
> t1,t2,t3 are just the lengths of the strings being concatented.
> Grow(x)=>y means Grow allocated x bytes at address y.
>
> So now, these two addresses 0x2aaaaaaab000 and 0x2b1e45256000  are  
> very far apart, like 400gig.
> And it seems the heap allocator wants to allocate an array to  
> describe the pages.

Ah, yes, that is most unfortunate.


> Hm. Page size is no longer tied to the underlying system -- no  
> longer vm-tied gc.
> Perhaps perhaps blowing it up, to say, 1meg, will address this?

I am working on minimizing the need for the global array, but we do  
need something that can be easily indexed like this.  Perhaps we need  
to pass a hint to RTOS.GetMemory that will try to allocate its regions  
close together.

> But really, an array to describe pages spanning the results of  
> separate memory allocations, seems wrong.
> A sparser data structure would be good, that could describe  
> arbitrary sized runs of pages as being in the same state.

Indeed.  As mentioned above, I am working to eliminate the need for  
this.  Code that starts us on this path will be checked in within a  
day or so.

>  - Jay
>
>
>
> From: jay.krell at cornell.edu
> To: m3devel at elegosoft.com
> Date: Sun, 16 Nov 2008 15:31:25 +0000
> Subject: [M3devel] duplicated code?
>
> Anyone want to clean up this duplicity?
>
>
> D:\dev2\cm3.2>dir /s/b asttotype.m3  StubCode.m3
> D:\dev2\cm3.2\m3-comm\sharedobjgen\src\AstToType.m3
> D:\dev2\cm3.2\m3-comm\stubgen\src\AstToType.m3
> D:\dev2\cm3.2\m3-db\stablegen\src\AstToType.m3
> D:\dev2\cm3.2\m3-comm\sharedobjgen\src\StubCode.m3
> D:\dev2\cm3.2\m3-comm\stubgen\src\StubCode.m3
>
>
> somewhere in there, AMD64_LINUX tries to allocate a lot of memory,  
> and fails
> either there or soon thereafter.
> The garbage collector is working.
>
>  - Jay
>
>
>
>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081116/e6f957c5/attachment.html>

From jay.krell at cornell.edu  Mon Nov 17 09:39:48 2008
From: jay.krell at cornell.edu (Jay)
Date: Mon, 17 Nov 2008 08:39:48 +0000
Subject: [M3devel] AMD64_LINUX (again)
Message-ID: <COL101-W10D65D8CA3093DB21EB5D1E6130@phx.gbl>


ok, with the sbrk workaround now:
 
 
http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-std-AMD64_LINUX-d5.7.0.tar.gz
 You don't have to build anything.
 
http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-min-AMD64_LINUX-d5.7.0.tar.gz
  You probably want to run ./do-cm3-std.py buildship.
 
http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-AMD64_LINUX.gz
  You get to build your own of almost everything (./install-config.py && ./do-pkg.py m3cc buildship && ./do-cm3-front.py buildship && ./install-front.py && ./do-cm3-std.py buildship)
 
 
OR you can bootstrap from an existing other system, roughly:
 cd m3-sys/m3cc && cm3 -DM3CC_TARGET=AMD64_LINUX && cd scripts/python && ./do-cm3-front.py AMD64_LINUX boot && ./boot1.py AMD64_LINUX and copy /cm3-boot-AMD64_LINUX-1.tar.gz to target machine and extract it, cd into it, make, and then you have native cm3 and proceed to previous..
 
 
I'm able to run Juno, Cube, Calculator, Mentor.
 
Hm. formsedit crashes. Darn.
 
Program received signal SIGSEGV, Segmentation fault.[Switching to Thread 1140881760 (LWP 13270)]0x00002afc180b9d6a in ScrollerVBTClass__PaintViewWithShadows (M3_Ao3CED_v=Cannot access memory at address 0x144006bf8)    at ../src/lego/POSIX/ScrollerVBTClass.m3:325325         VBT.PaintTint(v, r, v.shadow.bg);(gdb) cContinuing.Cannot find user-level thread for LWP 13242: generic error(gdb) bt#0  0x00002afc180b9d6a in ScrollerVBTClass__PaintViewWithShadows (M3_Ao3CED_v=Cannot access memory at address 0x144006bf8
) at ../src/lego/POSIX/ScrollerVBTClass.m3:325Cannot fetch general-purpose registers for thread 1140881760: generic error(gdb)
 
 
still might be worth other people trying out..
 
 
 - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081117/d6aecb25/attachment.html>

From wagner at elegosoft.com  Mon Nov 17 10:10:43 2008
From: wagner at elegosoft.com (Olaf Wagner)
Date: Mon, 17 Nov 2008 10:10:43 +0100
Subject: [M3devel] The undefined ROOT problem, rebuilding cm3
In-Reply-To: <491CB84D.4000008@wichita.edu>
References: <491CB84D.4000008@wichita.edu>
Message-ID: <20081117101043.m08cyzrq8gkksco8@mail.elegosoft.com>

Quoting "Rodney M. Bates" <rodney.bates at wichita.edu>:

> Here is how the undefined ROOT problem happens when trying to rebuild
> cm3 with itself.
>
> I had been making the assumption that commands build, ship, clean, when
> supplied to the do-cm3-*.sh scripts, did the same things as -build,
> -ship, -clean supplied to the cm3 command.  Not so.
>
> build actually does cm3 -build -override, as do buildlocal and empty
> commands to the scripts. clean does cm3 -clean -override, as does cleanlocal.
> There is a buildglobal command that does -build without -override, but
> it also does a ship as well.  There appears to be no way to just build,
> without either overriding or shipping.
>
> So, I have been running do-cm3-*.sh with no argument (and sometimes build).
> This starts a chain of events that eventually leads to an installed libm3
> that always executes quake script bundle-ov.tmpl, and it requires that
> either M3BUNDLE or ROOT be defined, neither of which normally is.  This
> will happen trying to compile any program that uses libm3, which is just
> about everything.
>
> We should document this somewhere.

It seems that this time my email wasn't complete, for whatever reasons.
I was out of town for some days so I noticed only today.

Rodney, I did ask you if you could write up some paragraphs for the
README in scripts and the web pages that clarifies this matter so that
others won't run into it again?

I always seem to miss some crucial points when trying to explain the
build system myself...

Thanks in advance,

Olaf
-- 
Olaf Wagner -- elego Software Solutions GmbH
                Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany
phone: +49 30 23 45 86 96  mobile: +49 177 2345 869  fax: +49 30 23 45 86 95
    http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin
Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194



From wagner at elegosoft.com  Mon Nov 17 10:30:00 2008
From: wagner at elegosoft.com (Olaf Wagner)
Date: Mon, 17 Nov 2008 10:30:00 +0100
Subject: [M3devel] The undefined ROOT problem, rebuilding cm3
In-Reply-To: <491CB84D.4000008@wichita.edu>
References: <491CB84D.4000008@wichita.edu>
Message-ID: <20081117103000.ld2m2zsu8w48o4ck@mail.elegosoft.com>

Hm, now I seem to be the victim of mail truncation. I'll  try a top-quote
this time (our system administrator will investigate this). Third attempt:

It seems that this time my email wasn't complete, for whatever reasons.
I was out of town for some days so I noticed only today.

Rodney, I did ask you if you could write up some paragraphs for the
README in scripts and the web pages that clarifies this matter so that
others won't run into it again?

I always seem to miss some crucial points when trying to explain the
build system myself...

Thanks in advance,

Olaf

Quoting "Rodney M. Bates" <rodney.bates at wichita.edu>:

> Here is how the undefined ROOT problem happens when trying to rebuild
> cm3 with itself.
>
> I had been making the assumption that commands build, ship, clean, when
> supplied to the do-cm3-*.sh scripts, did the same things as -build,
> -ship, -clean supplied to the cm3 command.  Not so.
>
> build actually does cm3 -build -override, as do buildlocal and empty
> commands to the scripts. clean does cm3 -clean -override, as does cleanlocal.
> There is a buildglobal command that does -build without -override, but
> it also does a ship as well.  There appears to be no way to just build,
> without either overriding or shipping.
>
> So, I have been running do-cm3-*.sh with no argument (and sometimes build).
> This starts a chain of events that eventually leads to an installed libm3
> that always executes quake script bundle-ov.tmpl, and it requires that
> either M3BUNDLE or ROOT be defined, neither of which normally is.  This
> will happen trying to compile any program that uses libm3, which is just
> about everything.
>
> We should document this somewhere.



-- 
Olaf Wagner -- elego Software Solutions GmbH
                Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany
phone: +49 30 23 45 86 96  mobile: +49 177 2345 869  fax: +49 30 23 45 86 95
    http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin
Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194



From jay.krell at cornell.edu  Mon Nov 17 10:35:18 2008
From: jay.krell at cornell.edu (Jay)
Date: Mon, 17 Nov 2008 09:35:18 +0000
Subject: [M3devel] gc vs. large spans of memory
In-Reply-To: <4B6BDC53-DB07-4498-A0D4-CB383DFFD487@cs.purdue.edu>
References: <COL101-W82BC319326E633D918F12DE6100@phx.gbl>
	<COL101-W16C880C91AE709A4E8463FE6100@phx.gbl> 
	<4B6BDC53-DB07-4498-A0D4-CB383DFFD487@cs.purdue.edu>
Message-ID: <BAY120-W65F83A8674A6D4949CA7CE6130@phx.gbl>


1meg pages weren't large enough to avoid running out of memory.
 
 
Any thoughts on using the boehm-gc?
I don't know if it is compacting.
Or even generational.
Or anything about it really, just that it is much ported and presumably well maintained, since it is part of "gcc", part of the gcc Java support.
 
 
Hm... I assume the right data structure here is a multi level array, that you index by picking off progressively less significant bits of an address (or page number) just like the hardware page tables for virtual memory (but without the perf benefits of a cache and using physical addresses after the first level).
When an entire level is at the same state, due to skipping a large span, you just have one shared entry for all of them, major savings.
 
 
Either that, or a binary searched array where you only store entries for state changes.
You'd use like the STL's upper_bound/lower_bound/equal_range functions, that is, really the same thing as bsearch, but when an entry isn't found, you return where it would be inserted, which is like the last place you looked before giving up.
 
It might be worth confirming "that I'm not crazy" -- that mmap behaves this way for other people, or that skimming its code suggests it is not surprising. Could be related to how malloc works also.
 
 - Jay



From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 16 Nov 2008 20:14:55 -0600CC: m3devel at elegosoft.comSubject: Re: [M3devel] duplicated code?


On 16 Nov 2008, at 10:18, Jay wrote:


Er, the large allocation does come from the gc itself.

What's being allocated?  The side array should not be too huge.

Tony, is there is an assumption that the heap is contiguous?That calls to RTOS.GetMemory return adjacent addresses?

No assumption.

This code allocates a large array:     IF desc = NIL OR newSideSpan # NUMBER(desc^) THEN      WITH newDesc = NEW(UNTRACED REF ARRAY OF Desc, newSideSpan) DOI'll know more shortly. I guess..it looks like the heap can be discontiguous, butwe do record keeping for what it all spans.

Correct.

The comments say:(* The array desc and the global variables p0, and p1 describe the pages   that are part of the traced heap.  Either p0 and p1 are equal to Nil and   no pages are allocated; or both are valid pages and page p is allocated   iff|          p0 <= p < p1|      AND desc[p - p0] != Unallocated Hm.. Grow (0x44000) => 0x2b1e45256000   total: 1.5MGetUntracedOpenArray(0x1a80)   span: 6.6M   density: 24%stubgen: Processing RemoteView.TGetUntracedOpenArray(0x3f0)t1:0xct2:0xat3:0x1Grow (0x52000) => 0x2aaaaaaab000   total: 1.8MGetUntracedOpenArray(0x1ce69fc8) I have GetUntracedOpenArray printing how many bytes it is asked for.t1,t2,t3 are just the lengths of the strings being concatented.Grow(x)=>y means Grow allocated x bytes at address y. So now, these two addresses 0x2aaaaaaab000 and 0x2b1e45256000  are very far apart, like 400gig.And it seems the heap allocator wants to allocate an array to describe the pages.

Ah, yes, that is most unfortunate.


Hm. Page size is no longer tied to the underlying system -- no longer vm-tied gc.Perhaps perhaps blowing it up, to say, 1meg, will address this?

I am working on minimizing the need for the global array, but we do need something that can be easily indexed like this.  Perhaps we need to pass a hint to RTOS.GetMemory that will try to allocate its regions close together.

But really, an array to describe pages spanning the results of separate memory allocations, seems wrong.A sparser data structure would be good, that could describe arbitrary sized runs of pages as being in the same state.

Indeed.  As mentioned above, I am working to eliminate the need for this.  Code that starts us on this path will be checked in within a day or so.

 - Jay

From: jay.krell at cornell.eduTo: m3devel at elegosoft.comDate: Sun, 16 Nov 2008 15:31:25 +0000Subject: [M3devel] duplicated code?Anyone want to clean up this duplicity?  D:\dev2\cm3.2>dir /s/b asttotype.m3  StubCode.m3D:\dev2\cm3.2\m3-comm\sharedobjgen\src\AstToType.m3D:\dev2\cm3.2\m3-comm\stubgen\src\AstToType.m3D:\dev2\cm3.2\m3-db\stablegen\src\AstToType.m3D:\dev2\cm3.2\m3-comm\sharedobjgen\src\StubCode.m3D:\dev2\cm3.2\m3-comm\stubgen\src\StubCode.m3 somewhere in there, AMD64_LINUX tries to allocate a lot of memory, and failseither there or soon thereafter.The garbage collector is working.  - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081117/8e6723b0/attachment.html>

From hosking at cs.purdue.edu  Mon Nov 17 13:21:26 2008
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Mon, 17 Nov 2008 06:21:26 -0600
Subject: [M3devel] gc vs. large spans of memory
In-Reply-To: <BAY120-W65F83A8674A6D4949CA7CE6130@phx.gbl>
References: <COL101-W82BC319326E633D918F12DE6100@phx.gbl>
	<COL101-W16C880C91AE709A4E8463FE6100@phx.gbl>
	<4B6BDC53-DB07-4498-A0D4-CB383DFFD487@cs.purdue.edu>
	<BAY120-W65F83A8674A6D4949CA7CE6130@phx.gbl>
Message-ID: <AE6DC073-2F37-437A-8712-0C5F09B5A97D@cs.purdue.edu>

On 17 Nov 2008, at 03:35, Jay wrote:

> 1meg pages weren't large enough to avoid running out of memory.

Is this still the sidespan array problem?

> Any thoughts on using the boehm-gc?

Hell no.

> I don't know if it is compacting.

Indeed, Boehm's collector does not do compaction.  Also, Boehm's is  
fully conservative ("ambiguous-roots + ambiguous heap"), though it is  
possible to configure it to use precise heap information (which CM3  
does provide).

> Or even generational.

It is generational, and has a parallel collection mode, but it is not  
concurrent like ours (we have a better chance of scaling on SMPs).   
Our collector will soon also be "on-the-fly", which means we won't  
need to have a stop-the-world phase where all the threads are stopped  
at the same time to initiate GC.  Instead, we will simply signal  
threads one at a time to prepare for GC.

> Or anything about it really, just that it is much ported and  
> presumably well maintained, since it is part of "gcc", part of the  
> gcc Java support.

Our collector is much ported and well-maintained too! ;-)  Seriously,  
I am quite sure we have no collector bugs currently (it has been  
running reliably for many years now).  The problems you are  
encountering are porting issues and not bugs in the collector.  Also,  
the current collector is written in Modula-3, and nicely integrated  
with the Modula-3 object model and run-time.   There is a *huge*  
amount to be said for eating your own dog-food by writing the  
collector in Modula-3.  There is *no* good reason to step outside  
Modula-3 to C as would be needed for Boehm GC.

> Hm... I assume the right data structure here is a multi level array,  
> that you index by picking off progressively less significant bits of  
> an address (or page number) just like the hardware page tables for  
> virtual memory (but without the perf benefits of a cache and using  
> physical addresses after the first level).
> When an entire level is at the same state, due to skipping a large  
> span, you just have one shared entry for all of them, major savings.

Right.  I have this partially implemented already.

> Either that, or a binary searched array where you only store entries  
> for state changes.
> You'd use like the STL's upper_bound/lower_bound/equal_range  
> functions, that is, really the same thing as bsearch, but when an  
> entry isn't found, you return where it would be inserted, which is  
> like the last place you looked before giving up.

I'm concerned about the lookup costs for this.

> It might be worth confirming "that I'm not crazy" -- that mmap  
> behaves this way for other people, or that skimming its code  
> suggests it is not surprising. Could be related to how malloc works  
> also.

Are there hints that can be passed to mmap on that platform to cause  
less scattered mappings?  It is odd that sbrk is less scattered, since  
it must also bottom out at mmap, but perhaps it is trying to maintain  
a reasonably compact allocation below the "brk".

>
>
>  - Jay
>
>
> From: hosking at cs.purdue.edu
> To: jay.krell at cornell.edu
> Date: Sun, 16 Nov 2008 20:14:55 -0600
> CC: m3devel at elegosoft.com
> Subject: Re: [M3devel] duplicated code?
>
>
> On 16 Nov 2008, at 10:18, Jay wrote:
>
> Er, the large allocation does come from the gc itself.
>
> What's being allocated?  The side array should not be too huge.
>
> Tony, is there is an assumption that the heap is contiguous?
> That calls to RTOS.GetMemory return adjacent addresses?
>
> No assumption.
>
> This code allocates a large array:
>
>     IF desc = NIL OR newSideSpan # NUMBER(desc^) THEN
>       WITH newDesc = NEW(UNTRACED REF ARRAY OF Desc, newSideSpan) DO
>
> I'll know more shortly.
>
> I guess..it looks like the heap can be discontiguous, but
> we do record keeping for what it all spans.
>
> Correct.
>
> The comments say:
> (* The array desc and the global variables p0, and p1 describe the  
> pages
>    that are part of the traced heap.  Either p0 and p1 are equal to  
> Nil and
>    no pages are allocated; or both are valid pages and page p is  
> allocated
>    iff
> |          p0 <= p < p1
> |      AND desc[p - p0] != Unallocated
>
>
> Hm..
>
> Grow (0x44000) => 0x2b1e45256000   total: 1.5M
> GetUntracedOpenArray(0x1a80)
>    span: 6.6M   density: 24%
> stubgen: Processing RemoteView.T
> GetUntracedOpenArray(0x3f0)
> t1:0xc
> t2:0xa
> t3:0x1
> Grow (0x52000) => 0x2aaaaaaab000   total: 1.8M
> GetUntracedOpenArray(0x1ce69fc8)
>
> I have GetUntracedOpenArray printing how many bytes it is asked for.
> t1,t2,t3 are just the lengths of the strings being concatented.
> Grow(x)=>y means Grow allocated x bytes at address y.
>
> So now, these two addresses 0x2aaaaaaab000 and 0x2b1e45256000  are  
> very far apart, like 400gig.
> And it seems the heap allocator wants to allocate an array to  
> describe the pages.
>
> Ah, yes, that is most unfortunate.
>
>
> Hm. Page size is no longer tied to the underlying system -- no  
> longer vm-tied gc.
> Perhaps perhaps blowing it up, to say, 1meg, will address this?
>
> I am working on minimizing the need for the global array, but we do  
> need something that can be easily indexed like this.  Perhaps we  
> need to pass a hint to RTOS.GetMemory that will try to allocate its  
> regions close together.
>
> But really, an array to describe pages spanning the results of  
> separate memory allocations, seems wrong.
> A sparser data structure would be good, that could describe  
> arbitrary sized runs of pages as being in the same state.
>
> Indeed.  As mentioned above, I am working to eliminate the need for  
> this.  Code that starts us on this path will be checked in within a  
> day or so.
>
>  - Jay
>
>
>
> From: jay.krell at cornell.edu
> To: m3devel at elegosoft.com
> Date: Sun, 16 Nov 2008 15:31:25 +0000
> Subject: [M3devel] duplicated code?
>
> Anyone want to clean up this duplicity?
>
>
> D:\dev2\cm3.2>dir /s/b asttotype.m3  StubCode.m3
> D:\dev2\cm3.2\m3-comm\sharedobjgen\src\AstToType.m3
> D:\dev2\cm3.2\m3-comm\stubgen\src\AstToType.m3
> D:\dev2\cm3.2\m3-db\stablegen\src\AstToType.m3
> D:\dev2\cm3.2\m3-comm\sharedobjgen\src\StubCode.m3
> D:\dev2\cm3.2\m3-comm\stubgen\src\StubCode.m3
>
>
> somewhere in there, AMD64_LINUX tries to allocate a lot of memory,  
> and fails
> either there or soon thereafter.
> The garbage collector is working.
>
>  - Jay
>
>
>
>
>
>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081117/70cfa516/attachment.html>

From jay.krell at cornell.edu  Mon Nov 17 13:48:59 2008
From: jay.krell at cornell.edu (Jay)
Date: Mon, 17 Nov 2008 12:48:59 +0000
Subject: [M3devel] gc vs. large spans of memory
In-Reply-To: <AE6DC073-2F37-437A-8712-0C5F09B5A97D@cs.purdue.edu>
References: <COL101-W82BC319326E633D918F12DE6100@phx.gbl>
	<COL101-W16C880C91AE709A4E8463FE6100@phx.gbl>
	<4B6BDC53-DB07-4498-A0D4-CB383DFFD487@cs.purdue.edu>
	<BAY120-W65F83A8674A6D4949CA7CE6130@phx.gbl>
	<AE6DC073-2F37-437A-8712-0C5F09B5A97D@cs.purdue.edu>
Message-ID: <COL101-W638E7324891BF366626DA4E6130@phx.gbl>


understood, ok.
 
I don't know the algorithm for allocating pages.
 
There is a "hint" parameter to mmap, it isn't exactly clear how it is handled.
Probably giving the block we have plus its size is reasonable.
I can try that out at some point, though using sbrk seems ok.
 
 - Jay



CC: m3devel at elegosoft.comFrom: hosking at cs.purdue.eduTo: jay.krell at cornell.eduSubject: Re: gc vs. large spans of memoryDate: Mon, 17 Nov 2008 06:21:26 -0600


On 17 Nov 2008, at 03:35, Jay wrote:


1meg pages weren't large enough to avoid running out of memory.

Is this still the sidespan array problem?

Any thoughts on using the boehm-gc?

Hell no.

I don't know if it is compacting.

Indeed, Boehm's collector does not do compaction.  Also, Boehm's is fully conservative ("ambiguous-roots + ambiguous heap"), though it is possible to configure it to use precise heap information (which CM3 does provide).

Or even generational.

It is generational, and has a parallel collection mode, but it is not concurrent like ours (we have a better chance of scaling on SMPs).  Our collector will soon also be "on-the-fly", which means we won't need to have a stop-the-world phase where all the threads are stopped at the same time to initiate GC.  Instead, we will simply signal threads one at a time to prepare for GC.

Or anything about it really, just that it is much ported and presumably well maintained, since it is part of "gcc", part of the gcc Java support.

Our collector is much ported and well-maintained too! ;-)  Seriously, I am quite sure we have no collector bugs currently (it has been running reliably for many years now).  The problems you are encountering are porting issues and not bugs in the collector.  Also, the current collector is written in Modula-3, and nicely integrated with the Modula-3 object model and run-time.   There is a *huge* amount to be said for eating your own dog-food by writing the collector in Modula-3.  There is *no* good reason to step outside Modula-3 to C as would be needed for Boehm GC.

Hm... I assume the right data structure here is a multi level array, that you index by picking off progressively less significant bits of an address (or page number) just like the hardware page tables for virtual memory (but without the perf benefits of a cache and using physical addresses after the first level).When an entire level is at the same state, due to skipping a large span, you just have one shared entry for all of them, major savings.

Right.  I have this partially implemented already.

Either that, or a binary searched array where you only store entries for state changes.You'd use like the STL's upper_bound/lower_bound/equal_range functions, that is, really the same thing as bsearch, but when an entry isn't found, you return where it would be inserted, which is like the last place you looked before giving up.

I'm concerned about the lookup costs for this.

It might be worth confirming "that I'm not crazy" -- that mmap behaves this way for other people, or that skimming its code suggests it is not surprising. Could be related to how malloc works also.

Are there hints that can be passed to mmap on that platform to cause less scattered mappings?  It is odd that sbrk is less scattered, since it must also bottom out at mmap, but perhaps it is trying to maintain a reasonably compact allocation below the "brk".

  - Jay

From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 16 Nov 2008 20:14:55 -0600CC: m3devel at elegosoft.comSubject: Re: [M3devel] duplicated code?


On 16 Nov 2008, at 10:18, Jay wrote:


Er, the large allocation does come from the gc itself.

What's being allocated?  The side array should not be too huge.

Tony, is there is an assumption that the heap is contiguous?That calls to RTOS.GetMemory return adjacent addresses?

No assumption.

This code allocates a large array:     IF desc = NIL OR newSideSpan # NUMBER(desc^) THEN      WITH newDesc = NEW(UNTRACED REF ARRAY OF Desc, newSideSpan) DOI'll know more shortly. I guess..it looks like the heap can be discontiguous, butwe do record keeping for what it all spans.

Correct.

The comments say:(* The array desc and the global variables p0, and p1 describe the pages   that are part of the traced heap.  Either p0 and p1 are equal to Nil and   no pages are allocated; or both are valid pages and page p is allocated   iff|          p0 <= p < p1|      AND desc[p - p0] != Unallocated Hm.. Grow (0x44000) => 0x2b1e45256000   total: 1.5MGetUntracedOpenArray(0x1a80)   span: 6.6M   density: 24%stubgen: Processing RemoteView.TGetUntracedOpenArray(0x3f0)t1:0xct2:0xat3:0x1Grow (0x52000) => 0x2aaaaaaab000   total: 1.8MGetUntracedOpenArray(0x1ce69fc8) I have GetUntracedOpenArray printing how many bytes it is asked for.t1,t2,t3 are just the lengths of the strings being concatented.Grow(x)=>y means Grow allocated x bytes at address y. So now, these two addresses 0x2aaaaaaab000 and 0x2b1e45256000  are very far apart, like 400gig.And it seems the heap allocator wants to allocate an array to describe the pages.

Ah, yes, that is most unfortunate.


Hm. Page size is no longer tied to the underlying system -- no longer vm-tied gc.Perhaps perhaps blowing it up, to say, 1meg, will address this?

I am working on minimizing the need for the global array, but we do need something that can be easily indexed like this.  Perhaps we need to pass a hint to RTOS.GetMemory that will try to allocate its regions close together.

But really, an array to describe pages spanning the results of separate memory allocations, seems wrong.A sparser data structure would be good, that could describe arbitrary sized runs of pages as being in the same state.

Indeed.  As mentioned above, I am working to eliminate the need for this.  Code that starts us on this path will be checked in within a day or so.

 - Jay

From: jay.krell at cornell.eduTo: m3devel at elegosoft.comDate: Sun, 16 Nov 2008 15:31:25 +0000Subject: [M3devel] duplicated code?Anyone want to clean up this duplicity?  D:\dev2\cm3.2>dir /s/b asttotype.m3  StubCode.m3D:\dev2\cm3.2\m3-comm\sharedobjgen\src\AstToType.m3D:\dev2\cm3.2\m3-comm\stubgen\src\AstToType.m3D:\dev2\cm3.2\m3-db\stablegen\src\AstToType.m3D:\dev2\cm3.2\m3-comm\sharedobjgen\src\StubCode.m3D:\dev2\cm3.2\m3-comm\stubgen\src\StubCode.m3 somewhere in there, AMD64_LINUX tries to allocate a lot of memory, and failseither there or soon thereafter.The garbage collector is working.  - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081117/4253a60f/attachment.html>

From jay.krell at cornell.edu  Mon Nov 17 14:00:01 2008
From: jay.krell at cornell.edu (Jay)
Date: Mon, 17 Nov 2008 13:00:01 +0000
Subject: [M3devel] gc vs. large spans of memory
In-Reply-To: <AE6DC073-2F37-437A-8712-0C5F09B5A97D@cs.purdue.edu>
References: <COL101-W82BC319326E633D918F12DE6100@phx.gbl>
	<COL101-W16C880C91AE709A4E8463FE6100@phx.gbl>
	<4B6BDC53-DB07-4498-A0D4-CB383DFFD487@cs.purdue.edu>
	<BAY120-W65F83A8674A6D4949CA7CE6130@phx.gbl>
	<AE6DC073-2F37-437A-8712-0C5F09B5A97D@cs.purdue.edu> 
Message-ID: <COL101-W25E6863F4A5BFE2141AA24E6130@phx.gbl>


 > 1 meg pages didn't suffice
 > Is this still the sidespan array problem?
 
I assume so.
I just tried it out, got the out of memory, and gave up on that approach.
I guess I could bump it up to 1 gig or more, just to prove the theory but
of course at some point I'm squeamish about wasting address space or actual memory.
1gig is tiny for address space but not for actual memory.
(assuming a full 64 bit address space, which actually is probably not available by far)
 
 - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081117/2160cca5/attachment.html>

From hosking at cs.purdue.edu  Mon Nov 17 19:31:01 2008
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Mon, 17 Nov 2008 12:31:01 -0600
Subject: [M3devel] gc vs. large spans of memory
In-Reply-To: <COL101-W25E6863F4A5BFE2141AA24E6130@phx.gbl>
References: <COL101-W82BC319326E633D918F12DE6100@phx.gbl>
	<COL101-W16C880C91AE709A4E8463FE6100@phx.gbl>
	<4B6BDC53-DB07-4498-A0D4-CB383DFFD487@cs.purdue.edu>
	<BAY120-W65F83A8674A6D4949CA7CE6130@phx.gbl>
	<AE6DC073-2F37-437A-8712-0C5F09B5A97D@cs.purdue.edu>
	<COL101-W25E6863F4A5BFE2141AA24E6130@phx.gbl>
Message-ID: <D67B5251-57BB-48F2-90E1-7F626B808FEA@cs.purdue.edu>

If sbrk works let's leave it at that for now.

On 17 Nov 2008, at 07:00, Jay wrote:

>  > 1 meg pages didn't suffice
>  > Is this still the sidespan array problem?
>
> I assume so.
> I just tried it out, got the out of memory, and gave up on that  
> approach.
> I guess I could bump it up to 1 gig or more, just to prove the  
> theory but
> of course at some point I'm squeamish about wasting address space or  
> actual memory.
> 1gig is tiny for address space but not for actual memory.
> (assuming a full 64 bit address space, which actually is probably  
> not available by far)
>
>  - Jay

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081117/c441cd1d/attachment.html>

From rodney.bates at wichita.edu  Sun Nov 23 19:06:18 2008
From: rodney.bates at wichita.edu (Rodney M. Bates)
Date: Sun, 23 Nov 2008 12:06:18 -0600
Subject: [M3devel] The undefined ROOT problem, rebuilding cm3
In-Reply-To: <20081117103000.ld2m2zsu8w48o4ck@mail.elegosoft.com>
References: <491CB84D.4000008@wichita.edu>
	<20081117103000.ld2m2zsu8w48o4ck@mail.elegosoft.com>
Message-ID: <49299B9A.7050103@wichita.edu>

Olaf Wagner wrote:
> Hm, now I seem to be the victim of mail truncation. I'll  try a top-quote
> this time (our system administrator will investigate this). Third attempt:
> 
> It seems that this time my email wasn't complete, for whatever reasons.
> I was out of town for some days so I noticed only today.
> 
> Rodney, I did ask you if you could write up some paragraphs for the
> README in scripts and the web pages that clarifies this matter so that
> others won't run into it again?

Done.

> 
> I always seem to miss some crucial points when trying to explain the
> build system myself...
> 
> Thanks in advance,
> 
> Olaf
> 
> Quoting "Rodney M. Bates" <rodney.bates at wichita.edu>:
> 
>> Here is how the undefined ROOT problem happens when trying to rebuild
>> cm3 with itself.
>>
>> I had been making the assumption that commands build, ship, clean, when
>> supplied to the do-cm3-*.sh scripts, did the same things as -build,
>> -ship, -clean supplied to the cm3 command.  Not so.
>>
>> build actually does cm3 -build -override, as do buildlocal and empty
>> commands to the scripts. clean does cm3 -clean -override, as does 
>> cleanlocal.
>> There is a buildglobal command that does -build without -override, but
>> it also does a ship as well.  There appears to be no way to just build,
>> without either overriding or shipping.
>>
>> So, I have been running do-cm3-*.sh with no argument (and sometimes build)



From michael.anderson at elego.de  Wed Nov 26 03:10:24 2008
From: michael.anderson at elego.de (Michael Anderson)
Date: Wed, 26 Nov 2008 03:10:24 +0100
Subject: [M3devel] compiling cvsup on Debian
Message-ID: <20081126031024.o8hbvsdkgo0kw884@mail.elegosoft.com>

Howdy there,

I'm trying to compile cvsup-snap-16.1h on Debian Etch with cm3, using  
this thread as
starting point:
https://mail.elegosoft.com/pipermail/m3devel/2007-June/000468.html

Thus far, I've:

- applied the patch
- changed line 744 of FileAttr.m3 to "tz: UNTRACED REF Utime.struct_timezone;"
- changed all IMPORT directives for "SchedulerPosix" to "Scheduler",  
as well as the
corresponding procedure calls as outlined here:

https://mail.elegosoft.com/pipermail/m3devel/2008-January/001210.html

Which that leaves me with just one teensy error:

"../src/SigHandler.m3", line 221: unknown qualification '.' (IOAlertWait)

which does go away if I change the IMPORT and procedure calls back to  
SchedulerPosix for
JUST SigHandler.m3.

Is it safe to use both "Scheduler" and "SchedulerPosix", or will  
Bad_Things happen?

Thanks,
mike


-- 
Michael Anderson
IT Services & Support

elego Software Solutions GmbH
Gustav-Meyer-Allee 25
Building 12.3 (BIG) room 227
13355 Berlin, Germany

phone +49 30 23 45 86 96      michael.anderson at elegosoft.com
fax   +49 30 23 45 86 95      http://www.elegosoft.com

Geschaeftsfuehrer: Olaf Wagner, Sitz Berlin
Amtsgericht Berlin-Charlottenburg, HRB 77719, USt-IdNr: DE163214194




From hosking at cs.purdue.edu  Wed Nov 26 17:03:52 2008
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Wed, 26 Nov 2008 10:03:52 -0600
Subject: [M3devel] compiling cvsup on Debian
In-Reply-To: <20081126031024.o8hbvsdkgo0kw884@mail.elegosoft.com>
References: <20081126031024.o8hbvsdkgo0kw884@mail.elegosoft.com>
Message-ID: <192844D9-5F3B-4E15-B2E9-C540D3EBD15B@cs.purdue.edu>

Should be just fine!

On 25 Nov 2008, at 20:10, Michael Anderson wrote:

> Howdy there,
>
> I'm trying to compile cvsup-snap-16.1h on Debian Etch with cm3,  
> using this thread as
> starting point:
> https://mail.elegosoft.com/pipermail/m3devel/2007-June/000468.html
>
> Thus far, I've:
>
> - applied the patch
> - changed line 744 of FileAttr.m3 to "tz: UNTRACED REF  
> Utime.struct_timezone;"
> - changed all IMPORT directives for "SchedulerPosix" to "Scheduler",  
> as well as the
> corresponding procedure calls as outlined here:
>
> https://mail.elegosoft.com/pipermail/m3devel/2008-January/001210.html
>
> Which that leaves me with just one teensy error:
>
> "../src/SigHandler.m3", line 221: unknown qualification  
> '.' (IOAlertWait)
>
> which does go away if I change the IMPORT and procedure calls back  
> to SchedulerPosix for
> JUST SigHandler.m3.
>
> Is it safe to use both "Scheduler" and "SchedulerPosix", or will  
> Bad_Things happen?
>
> Thanks,
> mike
>
>
> -- 
> Michael Anderson
> IT Services & Support
>
> elego Software Solutions GmbH
> Gustav-Meyer-Allee 25
> Building 12.3 (BIG) room 227
> 13355 Berlin, Germany
>
> phone +49 30 23 45 86 96      michael.anderson at elegosoft.com
> fax   +49 30 23 45 86 95      http://www.elegosoft.com
>
> Geschaeftsfuehrer: Olaf Wagner, Sitz Berlin
> Amtsgericht Berlin-Charlottenburg, HRB 77719, USt-IdNr: DE163214194
>
>



From rodney.bates at wichita.edu  Tue Nov  4 15:45:35 2008
From: rodney.bates at wichita.edu (Rodney M. Bates)
Date: Tue, 04 Nov 2008 08:45:35 -0600
Subject: [M3devel] New m3gdb documentation
Message-ID: <4910600F.7000009@wichita.edu>

I have new and greatly expanded documentation on m3gdb.  It is created
in docbook format, and I have managed to get it to convert to one-page
html.  I intend to do multi-page html and pdf also.

I am not sure where in the repository this should go, where to put html
files so they are available on the web, or what to do with mechanically
derived files in the repository.  I thought of creating a new subdirectory
doc/help/m3gdb, but I can only find how to get cvs to add a new ordinary
file, not a directory.

Also, I presume that Olaf will want to get the html to look like the other
pages, which some kind of style sheet should do, but I don't know where to
find that either.

Any help will be appreciated.

Rodney Bates


From wagner at elegosoft.com  Tue Nov  4 17:04:32 2008
From: wagner at elegosoft.com (Olaf Wagner)
Date: Tue, 04 Nov 2008 17:04:32 +0100
Subject: [M3devel] New m3gdb documentation
In-Reply-To: <4910600F.7000009@wichita.edu>
References: <4910600F.7000009@wichita.edu>
Message-ID: <20081104170432.1nn4yh39444sc48k@mail.elegosoft.com>

Quoting "Rodney M. Bates" <rodney.bates at wichita.edu>:

> I have new and greatly expanded documentation on m3gdb.  It is created
> in docbook format, and I have managed to get it to convert to one-page
> html.  I intend to do multi-page html and pdf also.

All documentation is checked-in below cm3/doc and cm3/www.
m3gdb should probably be integrated into doc.

> I am not sure where in the repository this should go, where to put html
> files so they are available on the web, or what to do with mechanically
> derived files in the repository.  I thought of creating a new subdirectory
> doc/help/m3gdb, but I can only find how to get cvs to add a new ordinary
> file, not a directory.

just use `cvs add <dir>'

> Also, I presume that Olaf will want to get the html to look like the other
> pages, which some kind of style sheet should do, but I don't know where to
> find that either.

The style sheet is normal.css at the top level.
You can reference it like this in the HTML header:

     <LINK HREF="normal.css" REL="stylesheet" TYPE="text/css">

It's checked in at cm3/www/normal.css.

When you've checked-in all your additions, we need to ship them
to the WWW server to make them visible. I can do that if you like,
just let me know when you're finished.

It's great that you're taking care of m3gdb and thus providing a
working debugger for all M3 users!

Regards,

Olaf

> Any help will be appreciated.
>
> Rodney Bates
-- 
Olaf Wagner -- elego Software Solutions GmbH
                Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany
phone: +49 30 23 45 86 96  mobile: +49 177 2345 869  fax: +49 30 23 45 86 95
    http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin
Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194



From rodney.bates at wichita.edu  Thu Nov  6 17:31:23 2008
From: rodney.bates at wichita.edu (Rodney M. Bates)
Date: Thu, 06 Nov 2008 10:31:23 -0600
Subject: [M3devel] New m3gdb documentation
Message-ID: <49131BDB.20903@wichita.edu>

There is a new and greatly expanded web page with documentation on m3gdb,
the Modula-3 debugger, located at:

   http://modula3.elegosoft.com/cm3/doc/help/m3gdb/m3gdb-onepage.html

It's also accessible with more indexing, through the normal Modula-3
web pages, for example, starting at:

   http://modula3.elegosoft.com/cm3

and clicking through:

   -> Documentation -> CM3 Operations Guide-> m3gdb

Rodney M. Bates




From jay.krell at cornell.edu  Sat Nov  8 16:40:50 2008
From: jay.krell at cornell.edu (Jay)
Date: Sat, 8 Nov 2008 15:40:50 +0000
Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
Message-ID: <COL101-W11F245FB5F2B72E83291B8E6180@phx.gbl>


Tony, how about switching these #if's?
I explain some of why below.

parse.c:

static void
m3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T)
{
  tree val;
#if 0
  if (o != 0 || TREE_TYPE (v) != dst_t) {
    v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),
     bitsize_int (o));
  }
#else
  /* failsafe, but inefficient */
  if (o != 0 || TREE_TYPE (v) != dst_t) {
    v = m3_build1 (ADDR_EXPR, t_addr, v);
    v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));
    v = m3_build1 (INDIRECT_REF, dst_t,
     m3_cast (build_pointer_type (dst_t), v));
  }
#endif
  TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */
  val = m3_cast (src_t, EXPR_REF (-1));
  if (src_T != dst_T) {
    val = m3_build1 (CONVERT_EXPR, dst_t, val);
  }
  add_stmt (build2 (MODIFY_EXPR, dst_t, v, val));
  EXPR_POP ();
}

m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T)
{
#if 0
  if (o != 0 || TREE_TYPE (v) != src_t) {
    v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),
     bitsize_int (o));
  }
#else
  /* failsafe, but inefficient */
  if (o != 0 || TREE_TYPE (v) != src_t) {
    v = m3_build1 (ADDR_EXPR, t_addr, v);
    v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));
    v = m3_build1 (INDIRECT_REF, src_t,
     m3_cast (build_pointer_type (src_t), v));
  }
#endif
  TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */
  if (src_T != dst_T) {
    v = m3_build1 (CONVERT_EXPR, dst_t, v);
  }
  EXPR_PUSH (v);
}

I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).

In an early version it seems that many loads/stores of global variables is wrong.

The first code affected is RTLinker_ExpandModuleTable.

PROCEDURE ExpandModuleTable () =
  CONST InitialTableSize = 500;
  VAR new_mods: ADDRESS;  n_bytes: INTEGER;
  BEGIN
    IF (modules = NIL) THEN
      (* first time... *)
      max_modules := InitialTableSize;
      modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr));
      IF (modules = NIL) THEN Cstdlib.abort (); END;
    ELSE
      n_bytes := max_modules * BYTESIZE (RT0.ModulePtr);
      new_mods := Cstdlib.malloc (n_bytes + n_bytes);
      IF (new_mods = NIL) THEN Cstdlib.abort (); END;
      EVAL Cstring.memcpy (new_mods, modules, n_bytes);
      Cstdlib.free (modules);
      modules := new_mods;
      INC (max_modules, max_modules);
    END;
  END ExpandModuleTable;

It should read back modules as NIL.
What happens is it reads back modules as not NIL,
and then max_modules should also be zero, but is "large", leading to malloc to fail,
and abort called.

The output of cm3cg -y is reasonable.
It indicates offset loads.

What is incorrect is that in many loads and stores, but not all, the
offsets are lost.

(923) begin_procedure
  procedure RTLinker__ExpandModuleTable
 RTLinker__ExpandModuleTable(924) set_source_line
  source line  208
(925) load_nil
(926) store
  store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb
(927) set_source_line
  source line  207
(928) set_source_line
  source line  210
(929) load_nil
(930) load
  m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xb

Here is the incorrect code:

 .globl RTLinker__ExpandModuleTable
 .stabd 46,0,0
 .stabn 68,0,206,.LM108
.LM108:
.LFBB8:
 .set nomips16
 .ent RTLinker__ExpandModuleTable
RTLinker__ExpandModuleTable:
 .frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0
 .mask 0xd0000000,-8
 .fmask 0x00000000,0
 daddiu $sp,$sp,-64
 sd $31,56($sp)
 sd $fp,48($sp)
 sd $28,40($sp)
 move $fp,$sp
 lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
 daddu $28,$28,$25
 daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
.LBB9:
 .stabn 68,0,208,.LM109
.LM109:
 sd $0,16($fp)
 .stabn 68,0,210,.LM110
.LM110:
 dla $2,MM_RTLinker
 ld $2,0($2)      << WRONG
 bne $2,$0,.L61
 .stabn 68,0,212,.LM111
.LM111:
 dla $3,MM_RTLinker
 ld $2,128($3)
 andi $2,$2,0x0
 ori $2,$2,0x1f4
 sd $2,128($3)
 .stabn 68,0,213,.LM112
.LM112:
 li $4,4000   # 0xfa0
 jal malloc
 sd $2,0($fp)
 ld $4,0($fp)
 dla $3,MM_RTLinker
 ld $2,136($3)
 andi $2,$2,0x0
 or $2,$4,$2
 sd $2,136($3)     << OK, offset seems low, but it agrees with elsewhere 
 .stabn 68,0,214,.LM113
.LM113:
 dla $2,MM_RTLinker
 ld $2,0($2)       << WRONG 
 bne $2,$0,.L65
 jal abort
 b .L65
.L61:
 .stabn 68,0,216,.LM114
.LM114:
 dla $2,MM_RTLinker
 ld $2,0($2)        << WRONG 
 dsll $2,$2,3
 sd $2,8($fp)
 .stabn 68,0,217,.LM115
.LM115:
 ld $3,8($fp)
 ld $2,8($fp)
 daddu $2,$3,$2
 move $4,$2
 jal malloc
 sd $2,0($fp)
 ld $2,0($fp)
 sd $2,16($fp)
 .stabn 68,0,218,.LM116
.LM116:
 ld $2,16($fp)
 bne $2,$0,.L64
 jal abort
.L64:
 .stabn 68,0,219,.LM117
.LM117:
 ld $4,16($fp)
 dla $2,MM_RTLinker
 ld $2,0($2)          << WRONG
 ld $3,8($fp)
 move $5,$2
 move $6,$3
 jal memcpy
 sd $2,0($fp)
 .stabn 68,0,220,.LM118
.LM118:
 dla $2,MM_RTLinker
 ld $2,0($2)           << WRONG
 move $4,$2
 jal free
 .stabn 68,0,221,.LM119
.LM119:
 ld $4,16($fp)
 dla $3,MM_RTLinker
 ld $2,136($3)         << OK
 andi $2,$2,0x0
 or $2,$4,$2
 sd $2,136($3)         << OK 
 .stabn 68,0,222,.LM120
.LM120:
 dla $2,MM_RTLinker
 ld $3,0($2)           << WRONG
 dla $2,MM_RTLinker
 ld $2,0($2)           << WRONG
 daddu $4,$3,$2
 dla $3,MM_RTLinker
 ld $2,128($3)         << OK
 andi $2,$2,0x0
 or $2,$2,$4
 sd $2,128($3)         << OK
.L65:
.LBE9:
 .stabn 68,0,224,.LM121
.LM121:
 move $sp,$fp
 ld $31,56($sp)
 ld $fp,48($sp)
 ld $28,40($sp)
 daddiu $sp,$sp,64
 j $31
 .end RTLinker__ExpandModuleTable

Here is corrected code:

RTLinker__ExpandModuleTable:
 .frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0
 .mask 0xd0000000,-8
 .fmask 0x00000000,0
 daddiu $sp,$sp,-64
 sd $31,56($sp)
 sd $fp,48($sp)
 sd $28,40($sp)
 move $fp,$sp
 lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
 daddu $28,$28,$25
 daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
.LBB9:
 .stabn 68,0,208,.LM109
.LM109:
 sd $0,16($fp)
 .stabn 68,0,210,.LM110
.LM110:
 dla $2,MM_RTLinker
 daddiu $2,$2,136     << CORRECT (offset seems low, but at least it isn't zero) 
 ld $2,0($2)
 bne $2,$0,.L61
 .stabn 68,0,212,.LM111
.LM111:
 dla $2,MM_RTLinker
 daddiu $3,$2,128     << CORRECT 
 li $2,500   # 0x1f4
 sd $2,0($3)
 .stabn 68,0,213,.LM112
.LM112:
 li $4,4000   # 0xfa0
 jal malloc
 sd $2,0($fp)
 dla $2,MM_RTLinker
 daddiu $3,$2,136      << CORRECT 
 ld $2,0($fp)
 sd $2,0($3)
 .stabn 68,0,214,.LM113
.LM113:
 dla $2,MM_RTLinker
 daddiu $2,$2,136      << CORRECT 
 ld $2,0($2)
 bne $2,$0,.L65
 jal abort
 b .L65
.L61:

I still dump core with this change, but I get much further.
I have not tested this change at all otherwise -- i.e. on working platforms.

I have not debugged through the gcc/m3cg code to see why
the offsets are only sometimes lost, or why other platforms
have no problem. Is MIPS's "global pointer" unusual these days?
I know SPARC has similar, and all the non-x86, non-AMD64 NT platforms
had similar, but that's essentially nothing now (I'm not counting IA64).
I know Linux has PLT (procedure linkage table) and GOT (global offset table).
Same thing?
I haven't looked much at other MIPS ports yet.

http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8

is what made it look about like it does -- in particularing using BIT_FIELD_REF.

1.7:

static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */
     tree v;
     int o;
     tree src_t, dest_t;
     m3_type src_T, dest_T;
{
  if (o == 0 && TREE_TYPE (v) == src_t) {
    EXPR_PUSH (v);
  } else {
    tree adr = m3_build1 (ADDR_EXPR, t_addr, v);
    if (o != 0) {
      adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT));
    }
    EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t,
                          m3_cast (build_pointer_type (src_t), adr)));
  }
#if 1
  if (src_T != dest_T) {
    EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1));
  }
  if (debug_vars) {
    const char *name = "noname";
    if (v != 0 && DECL_NAME(v) != 0) {
      name = IDENTIFIER_POINTER(DECL_NAME(v));
    }
    fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n", name,
            o, src_T, dest_T);
  }
#else
  if (src_T != dest_T) {
    if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) {
      EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1));
    } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) {
      EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1));
    } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) {
      /* okay */
    } else {
      fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n",
                  src_T, dest_T);
    }
  }
#endif
}
static void
m3_store (v, o, src_t, dest_t)  /* GCC32OK */
     tree v;
     int o;
     tree src_t, dest_t;
{
  tree lhs, rhs;
  if (TREE_TYPE (EXPR_REF (-1)) == src_t) {
    rhs = EXPR_REF (-1);
  } else {
    rhs = m3_cast (src_t, EXPR_REF (-1));
  }
  if (o == 0 && TREE_TYPE (v) == dest_t) {
    lhs = v;
  } else {
    tree f = make_node (FIELD_DECL);
    TREE_TYPE (f) = dest_t;
    DECL_ALIGN (f) = TYPE_ALIGN (dest_t);
    DECL_SIZE (f) = TYPE_SIZE (dest_t);
    DECL_MODE (f) = TYPE_MODE (dest_t);
    DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT);
    DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT);
    DECL_FIELD_CONTEXT (f) = TREE_TYPE (v);
    lhs = m3_build2 (COMPONENT_REF, dest_t, v, f);
  }
#if 1
    expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);
#else
  if (dest_t ==  NULL) {
    expand_assignment (lhs, rhs, 0, 0);
  } else {
    expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);
  }
#endif
  EXPR_POP ();
}

1.8:
static void
m3_load (tree v, int o,
  tree src_t, m3_type src_T,
  tree dst_t, m3_type dst_T) /* GCC32OK */
{
  if (o == 0 && TREE_TYPE (v) == src_t) {
    EXPR_PUSH (v);
  } else {
    EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),
     bitsize_int (o)));
  }
  if (src_T != dst_T) {
    EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1));
  }
  if (option_vars_trace) {
    const char *name = "noname";
    if (v != 0 && DECL_NAME(v) != 0) {
      name = IDENTIFIER_POINTER(DECL_NAME(v));
    }
    fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n", name,
     o, src_T, dst_T);
  }
}
static void
m3_store (tree v, int o, tree src_t, tree dst_t)  /* GCC32OK */
{
  tree lhs, rhs;
  if (TREE_TYPE (EXPR_REF (-1)) == src_t) {
    rhs = EXPR_REF (-1);
  } else {
    rhs = m3_cast (src_t, EXPR_REF (-1));
  }
  if (o == 0 && TREE_TYPE (v) == dst_t) {
    lhs = v;
  } else {
    lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),
       bitsize_int (o));
  }
  if (src_t != dst_t) {
    rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs);
  }
  expand_assignment (lhs, rhs, 0, 0);
  EXPR_POP ();
}

You should be able to repro this by configuring m3cg for mips64-openbsd.
I've only tested on cygwin host, but I could check others..
 
Thanks,
 - Jay

From jay.krell at cornell.edu  Sat Nov  8 16:45:15 2008
From: jay.krell at cornell.edu (Jay)
Date: Sat, 8 Nov 2008 15:45:15 +0000
Subject: [M3devel] FW: m3_load/m3_store/bit_field_ref lose offset??
Message-ID: <COL101-W3DB0BA622C4D31A82B602E6180@phx.gbl>


truncated as usual...along with losing a lot of whitespace..
Let's see...


Tony, how about switching these #if's?
I explain some of why below.

static void
m3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T)
{
  tree val;
#if 0
  if (o != 0 || TREE_TYPE (v) != dst_t) {
    v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),
     bitsize_int (o));
  }
#else
  /* failsafe, but inefficient */
  if (o != 0 || TREE_TYPE (v) != dst_t) {
    v = m3_build1 (ADDR_EXPR, t_addr, v);
    v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));
    v = m3_build1 (INDIRECT_REF, dst_t,
     m3_cast (build_pointer_type (dst_t), v));
  }
#endif
  TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */
  val = m3_cast (src_t, EXPR_REF (-1));
  if (src_T != dst_T) {
    val = m3_build1 (CONVERT_EXPR, dst_t, val);
  }
  add_stmt (build2 (MODIFY_EXPR, dst_t, v, val));
  EXPR_POP ();
}
m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T)
{
#if 0
  if (o != 0 || TREE_TYPE (v) != src_t) {
    v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),
     bitsize_int (o));
  }
#else
  /* failsafe, but inefficient */
  if (o != 0 || TREE_TYPE (v) != src_t) {
    v = m3_build1 (ADDR_EXPR, t_addr, v);
    v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));
    v = m3_build1 (INDIRECT_REF, src_t,
     m3_cast (build_pointer_type (src_t), v));
  }
#endif
  TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */
  if (src_T != dst_T) {
    v = m3_build1 (CONVERT_EXPR, dst_t, v);
  }
  EXPR_PUSH (v);
}

I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).
In an early version it seems that many loads/stores of global variables is wrong.
The "first" code affected is RTLinker_ExpandModuleTable.

PROCEDURE ExpandModuleTable () =
  CONST InitialTableSize = 500;
  VAR new_mods: ADDRESS;  n_bytes: INTEGER;
  BEGIN
    IF (modules = NIL) THEN
      (* first time... *)
      max_modules := InitialTableSize;
      modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr));
      IF (modules = NIL) THEN Cstdlib.abort (); END;
    ELSE
      n_bytes := max_modules * BYTESIZE (RT0.ModulePtr);
      new_mods := Cstdlib.malloc (n_bytes + n_bytes);
      IF (new_mods = NIL) THEN Cstdlib.abort (); END;
      EVAL Cstring.memcpy (new_mods, modules, n_bytes);
      Cstdlib.free (modules);
      modules := new_mods;
      INC (max_modules, max_modules);
    END;
  END ExpandModuleTable;

It should read back modules as NIL.
What happens is it reads back modules as not NIL,
and then max_modules should also be zero, but is "large", leading to malloc to fail,
and abort called.

The output of cm3cg -y is reasonable.
It indicates offset loads.

What is incorrect is that in many loads and stores, but not all, the
offsets are lost.

(923) begin_procedure
  procedure RTLinker__ExpandModuleTable
 RTLinker__ExpandModuleTable(924) set_source_line
  source line  208
(925) load_nil
(926) store
  store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb
(927) set_source_line
  source line  207
(928) set_source_line
  source line  210
(929) load_nil
(930) load
  m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xb

Here is the incorrect code:

 .globl RTLinker__ExpandModuleTable
 .stabd 46,0,0
 .stabn 68,0,206,.LM108
.LM108:
.LFBB8:
 .set nomips16
 .ent RTLinker__ExpandModuleTable
RTLinker__ExpandModuleTable:
 .frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0
 .mask 0xd0000000,-8
 .fmask 0x00000000,0
 daddiu $sp,$sp,-64
 sd $31,56($sp)
 sd $fp,48($sp)
 sd $28,40($sp)
 move $fp,$sp
 lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
 daddu $28,$28,$25
 daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
.LBB9:
 .stabn 68,0,208,.LM109
.LM109:
 sd $0,16($fp)
 .stabn 68,0,210,.LM110
.LM110:
 dla $2,MM_RTLinker
 ld $2,0($2)      << WRONG
 bne $2,$0,.L61
 .stabn 68,0,212,.LM111
.LM111:
 dla $3,MM_RTLinker
 ld $2,128($3)
 andi $2,$2,0x0
 ori $2,$2,0x1f4
 sd $2,128($3)
 .stabn 68,0,213,.LM112
.LM112:
 li $4,4000   # 0xfa0
 jal malloc
 sd $2,0($fp)
 ld $4,0($fp)
 dla $3,MM_RTLinker
 ld $2,136($3)
 andi $2,$2,0x0
 or $2,$4,$2
 sd $2,136($3)     << OK, offset seems low, but it agrees with elsewhere 
 .stabn 68,0,214,.LM113
.LM113:
 dla $2,MM_RTLinker
 ld $2,0($2)       << WRONG 
 bne $2,$0,.L65
 jal abort
 b .L65
.L61:
 .stabn 68,0,216,.LM114
.LM114:
 dla $2,MM_RTLinker
 ld $2,0($2)        << WRONG 
 dsll $2,$2,3
 sd $2,8($fp)
 .stabn 68,0,217,.LM115
.LM115:
 ld $3,8($fp)
 ld $2,8($fp)
 daddu $2,$3,$2
 move $4,$2
 jal malloc
 sd $2,0($fp)
 ld $2,0($fp)
 sd $2,16($fp)
 .stabn 68,0,218,.LM116
.LM116:
 ld $2,16($fp)
 bne $2,$0,.L64
 jal abort
.L64:
 .stabn 68,0,219,.LM117
.LM117:
 ld $4,16($fp)
 dla $2,MM_RTLinker
 ld $2,0($2)          << WRONG
 ld $3,8($fp)
 move $5,$2
 move $6,$3
 jal memcpy
 sd $2,0($fp)
 .stabn 68,0,220,.LM118
.LM118:
 dla $2,MM_RTLinker
 ld $2,0($2)           << WRONG
 move $4,$2
 jal free
 .stabn 68,0,221,.LM119
.LM119:
 ld $4,16($fp)
 dla $3,MM_RTLinker
 ld $2,136($3)         << OK
 andi $2,$2,0x0
 or $2,$4,$2
 sd $2,136($3)         << OK 
 .stabn 68,0,222,.LM120
.LM120:
 dla $2,MM_RTLinker
 ld $3,0($2)           << WRONG
 dla $2,MM_RTLinker
 ld $2,0($2)           << WRONG
 daddu $4,$3,$2
 dla $3,MM_RTLinker
 ld $2,128($3)         << OK
 andi $2,$2,0x0
 or $2,$2,$4
 sd $2,128($3)         << OK
.L65:
.LBE9:
 .stabn 68,0,224,.LM121
.LM121:
 move $sp,$fp
 ld $31,56($sp)
 ld $fp,48($sp)
 ld $28,40($sp)
 daddiu $sp,$sp,64
 j $31
 .end RTLinker__ExpandModuleTable

Here is corrected code:

RTLinker__ExpandModuleTable:
 .frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0
 .mask 0xd0000000,-8
 .fmask 0x00000000,0
 daddiu $sp,$sp,-64
 sd $31,56($sp)
 sd $fp,48($sp)
 sd $28,40($sp)
 move $fp,$sp
 lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
 daddu $28,$28,$25
 daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
.LBB9:
 .stabn 68,0,208,.LM109
.LM109:
 sd $0,16($fp)
 .stabn 68,0,210,.LM110
.LM110:
 dla $2,MM_RTLinker
 daddiu $2,$2,136     << CORRECT (offset seems low, but at least it isn't zero) 
 ld $2,0($2)
 bne $2,$0,.L61
 .stabn 68,0,212,.LM111
.LM111:
 dla $2,MM_RTLinker
 daddiu $3,$2,128     << CORRECT 
 li $2,500   # 0x1f4
 sd $2,0($3)
 .stabn 68,0,213,.LM112
.LM112:
 li $4,4000   # 0xfa0
 jal malloc
 sd $2,0($fp)
 dla $2,MM_RTLinker
 daddiu $3,$2,136      << CORRECT 
 ld $2,0($fp)
 sd $2,0($3)
 .stabn 68,0,214,.LM113
.LM113:
 dla $2,MM_RTLinker
 daddiu $2,$2,136      << CORRECT 
 ld $2,0($2)
 bne $2,$0,.L65
 jal abort
 b .L65
.L61:

I still dump core with this change, but I get much further.
I have not tested this change at all otherwise -- i.e. on working platforms.

I have not debugged through the gcc/m3cg code to see why
the offsets are only sometimes lost, or why other platforms
have no problem. Is MIPS's "global pointer" unusual these days?
I know SPARC has similar, and all the non-x86, non-AMD64 NT platforms
had similar, but that's essentially nothing now (I'm not counting IA64).
I know Linux has PLT (procedure linkage table) and GOT (global offset table).
Same thing?
I haven't looked much at other MIPS ports yet.

http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8

is what made it look about like it does -- in particularing using BIT_FIELD_REF.

1.7:

static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */
     tree v;
     int o;
     tree src_t, dest_t;
     m3_type src_T, dest_T;
{
  if (o == 0 && TREE_TYPE (v) == src_t) {
    EXPR_PUSH (v);
  } else {
    tree adr = m3_build1 (ADDR_EXPR, t_addr, v);
    if (o != 0) {
      adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT));
    }
    EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t,
                          m3_cast (build_pointer_type (src_t), adr)));
  }
#if 1
  if (src_T != dest_T) {
    EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1));
  }
  if (debug_vars) {
    const char *name = "noname";
    if (v != 0 && DECL_NAME(v) != 0) {
      name = IDENTIFIER_POINTER(DECL_NAME(v));
    }
    fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n", name,
            o, src_T, dest_T);
  }
#else
  if (src_T != dest_T) {
    if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) {
      EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1));
    } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) {
      EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1));
    } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) {
      /* okay */
    } else {
      fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n",
                  src_T, dest_T);
    }
  }
#endif
}
static void
m3_store (v, o, src_t, dest_t)  /* GCC32OK */
     tree v;
     int o;
     tree src_t, dest_t;
{
  tree lhs, rhs;
  if (TREE_TYPE (EXPR_REF (-1)) == src_t) {
    rhs = EXPR_REF (-1);
  } else {
    rhs = m3_cast (src_t, EXPR_REF (-1));
  }
  if (o == 0 && TREE_TYPE (v) == dest_t) {
    lhs = v;
  } else {
    tree f = make_node (FIELD_DECL);
    TREE_TYPE (f) = dest_t;
    DECL_ALIGN (f) = TYPE_ALIGN (dest_t);
    DECL_SIZE (f) = TYPE_SIZE (dest_t);
    DECL_MODE (f) = TYPE_MODE (dest_t);
    DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT);
    DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT);
    DECL_FIELD_CONTEXT (f) = TREE_TYPE (v);
    lhs = m3_build2 (COMPONENT_REF, dest_t, v, f);
  }
#if 1
    expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);
#else
  if (dest_t ==  NULL) {
    expand_assignment (lhs, rhs, 0, 0);
  } else {
    expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);
  }
#endif
  EXPR_POP ();
}

1.8:
static void
m3_load (tree v, int o,
  tree src_t, m3_type src_T,
  tree dst_t, m3_type dst_T) /* GCC32OK */
{
  if (o == 0 && TREE_TYPE (v) == src_t) {
    EXPR_PUSH (v);
  } else {
    EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),
     bitsize_int (o)));
  }
  if (src_T != dst_T) {
    EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1));
  }
  if (option_vars_trace) {
    const char *name = "noname";
    if (v != 0 && DECL_NAME(v) != 0) {
      name = IDENTIFIER_POINTER(DECL_NAME(v));
    }
    fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n", name,
     o, src_T, dst_T);
  }
}
static void
m3_store (tree v, int o, tree src_t, tree dst_t)  /* GCC32OK */
{
  tree lhs, rhs;
  if (TREE_TYPE (EXPR_REF (-1)) == src_t) {
    rhs = EXPR_REF (-1);
  } else {
    rhs = m3_cast (src_t, EXPR_REF (-1));
  }
  if (o == 0 && TREE_TYPE (v) == dst_t) {
    lhs = v;
  } else {
    lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),
       bitsize_int (o));
  }
  if (src_t != dst_t) {
    rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs);
  }
  expand_assignment (lhs, rhs, 0, 0);
  EXPR_POP ();
}

 - Jay

From hosking at cs.purdue.edu  Sat Nov  8 23:55:54 2008
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Sat, 8 Nov 2008 22:55:54 +0000
Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
In-Reply-To: <COL101-W3DB0BA622C4D31A82B602E6180@phx.gbl>
References: <COL101-W3DB0BA622C4D31A82B602E6180@phx.gbl>
Message-ID: <C47F6B5A-F154-4485-B839-96185B59E3B6@cs.purdue.edu>

Have you checked to make sure that the pre-gcc IR makes sense?  Does  
it see these values as 32-bit or 64-bit?  The reason I ask is that  
there may be something wrong in your m3middle target specs that cause  
bad IR.  Once we know the IR looks right then we can try and fix the  
problem.

Antony Hosking | Associate Professor | Computer Science | Purdue  
University
305 N. University Street | West Lafayette | IN 47907 | USA
Office +1 765 494 6001 | Mobile +1 765 427 5484



On 8 Nov 2008, at 15:45, Jay wrote:

>
> truncated as usual...along with losing a lot of whitespace..
> Let's see...
>
>
> Tony, how about switching these #if's?
> I explain some of why below.
>
> static void
> m3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t,  
> m3_type dst_T)
> {
>  tree val;
> #if 0
>  if (o != 0 || TREE_TYPE (v) != dst_t) {
>    v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),
>     bitsize_int (o));
>  }
> #else
>  /* failsafe, but inefficient */
>  if (o != 0 || TREE_TYPE (v) != dst_t) {
>    v = m3_build1 (ADDR_EXPR, t_addr, v);
>    v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));
>    v = m3_build1 (INDIRECT_REF, dst_t,
>     m3_cast (build_pointer_type (dst_t), v));
>  }
> #endif
>  TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing  
> problems */
>  val = m3_cast (src_t, EXPR_REF (-1));
>  if (src_T != dst_T) {
>    val = m3_build1 (CONVERT_EXPR, dst_t, val);
>  }
>  add_stmt (build2 (MODIFY_EXPR, dst_t, v, val));
>  EXPR_POP ();
> }
> m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t,  
> m3_type dst_T)
> {
> #if 0
>  if (o != 0 || TREE_TYPE (v) != src_t) {
>    v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),
>     bitsize_int (o));
>  }
> #else
>  /* failsafe, but inefficient */
>  if (o != 0 || TREE_TYPE (v) != src_t) {
>    v = m3_build1 (ADDR_EXPR, t_addr, v);
>    v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));
>    v = m3_build1 (INDIRECT_REF, src_t,
>     m3_cast (build_pointer_type (src_t), v));
>  }
> #endif
>  TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing  
> problems */
>  if (src_T != dst_T) {
>    v = m3_build1 (CONVERT_EXPR, dst_t, v);
>  }
>  EXPR_PUSH (v);
> }
>
> I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).
> In an early version it seems that many loads/stores of global  
> variables is wrong.
> The "first" code affected is RTLinker_ExpandModuleTable.
>
> PROCEDURE ExpandModuleTable () =
>  CONST InitialTableSize = 500;
>  VAR new_mods: ADDRESS;  n_bytes: INTEGER;
>  BEGIN
>    IF (modules = NIL) THEN
>      (* first time... *)
>      max_modules := InitialTableSize;
>      modules := Cstdlib.malloc (InitialTableSize * BYTESIZE  
> (RT0.ModulePtr));
>      IF (modules = NIL) THEN Cstdlib.abort (); END;
>    ELSE
>      n_bytes := max_modules * BYTESIZE (RT0.ModulePtr);
>      new_mods := Cstdlib.malloc (n_bytes + n_bytes);
>      IF (new_mods = NIL) THEN Cstdlib.abort (); END;
>      EVAL Cstring.memcpy (new_mods, modules, n_bytes);
>      Cstdlib.free (modules);
>      modules := new_mods;
>      INC (max_modules, max_modules);
>    END;
>  END ExpandModuleTable;
>
> It should read back modules as NIL.
> What happens is it reads back modules as not NIL,
> and then max_modules should also be zero, but is "large", leading to  
> malloc to fail,
> and abort called.
>
> The output of cm3cg -y is reasonable.
> It indicates offset loads.
>
> What is incorrect is that in many loads and stores, but not all, the
> offsets are lost.
>
> (923) begin_procedure
>  procedure RTLinker__ExpandModuleTable
> RTLinker__ExpandModuleTable(924) set_source_line
>  source line  208
> (925) load_nil
> (926) store
>  store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb
> (927) set_source_line
>  source line  207
> (928) set_source_line
>  source line  210
> (929) load_nil
> (930) load
>  m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xb
>
> Here is the incorrect code:
>
> .globl RTLinker__ExpandModuleTable
> .stabd 46,0,0
> .stabn 68,0,206,.LM108
> .LM108:
> .LFBB8:
> .set nomips16
> .ent RTLinker__ExpandModuleTable
> RTLinker__ExpandModuleTable:
> .frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0
> .mask 0xd0000000,-8
> .fmask 0x00000000,0
> daddiu $sp,$sp,-64
> sd $31,56($sp)
> sd $fp,48($sp)
> sd $28,40($sp)
> move $fp,$sp
> lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
> daddu $28,$28,$25
> daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
> .LBB9:
> .stabn 68,0,208,.LM109
> .LM109:
> sd $0,16($fp)
> .stabn 68,0,210,.LM110
> .LM110:
> dla $2,MM_RTLinker
> ld $2,0($2)      << WRONG
> bne $2,$0,.L61
> .stabn 68,0,212,.LM111
> .LM111:
> dla $3,MM_RTLinker
> ld $2,128($3)
> andi $2,$2,0x0
> ori $2,$2,0x1f4
> sd $2,128($3)
> .stabn 68,0,213,.LM112
> .LM112:
> li $4,4000   # 0xfa0
> jal malloc
> sd $2,0($fp)
> ld $4,0($fp)
> dla $3,MM_RTLinker
> ld $2,136($3)
> andi $2,$2,0x0
> or $2,$4,$2
> sd $2,136($3)     << OK, offset seems low, but it agrees with  
> elsewhere
> .stabn 68,0,214,.LM113
> .LM113:
> dla $2,MM_RTLinker
> ld $2,0($2)       << WRONG
> bne $2,$0,.L65
> jal abort
> b .L65
> .L61:
> .stabn 68,0,216,.LM114
> .LM114:
> dla $2,MM_RTLinker
> ld $2,0($2)        << WRONG
> dsll $2,$2,3
> sd $2,8($fp)
> .stabn 68,0,217,.LM115
> .LM115:
> ld $3,8($fp)
> ld $2,8($fp)
> daddu $2,$3,$2
> move $4,$2
> jal malloc
> sd $2,0($fp)
> ld $2,0($fp)
> sd $2,16($fp)
> .stabn 68,0,218,.LM116
> .LM116:
> ld $2,16($fp)
> bne $2,$0,.L64
> jal abort
> .L64:
> .stabn 68,0,219,.LM117
> .LM117:
> ld $4,16($fp)
> dla $2,MM_RTLinker
> ld $2,0($2)          << WRONG
> ld $3,8($fp)
> move $5,$2
> move $6,$3
> jal memcpy
> sd $2,0($fp)
> .stabn 68,0,220,.LM118
> .LM118:
> dla $2,MM_RTLinker
> ld $2,0($2)           << WRONG
> move $4,$2
> jal free
> .stabn 68,0,221,.LM119
> .LM119:
> ld $4,16($fp)
> dla $3,MM_RTLinker
> ld $2,136($3)         << OK
> andi $2,$2,0x0
> or $2,$4,$2
> sd $2,136($3)         << OK
> .stabn 68,0,222,.LM120
> .LM120:
> dla $2,MM_RTLinker
> ld $3,0($2)           << WRONG
> dla $2,MM_RTLinker
> ld $2,0($2)           << WRONG
> daddu $4,$3,$2
> dla $3,MM_RTLinker
> ld $2,128($3)         << OK
> andi $2,$2,0x0
> or $2,$2,$4
> sd $2,128($3)         << OK
> .L65:
> .LBE9:
> .stabn 68,0,224,.LM121
> .LM121:
> move $sp,$fp
> ld $31,56($sp)
> ld $fp,48($sp)
> ld $28,40($sp)
> daddiu $sp,$sp,64
> j $31
> .end RTLinker__ExpandModuleTable
>
> Here is corrected code:
>
> RTLinker__ExpandModuleTable:
> .frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0
> .mask 0xd0000000,-8
> .fmask 0x00000000,0
> daddiu $sp,$sp,-64
> sd $31,56($sp)
> sd $fp,48($sp)
> sd $28,40($sp)
> move $fp,$sp
> lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
> daddu $28,$28,$25
> daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
> .LBB9:
> .stabn 68,0,208,.LM109
> .LM109:
> sd $0,16($fp)
> .stabn 68,0,210,.LM110
> .LM110:
> dla $2,MM_RTLinker
> daddiu $2,$2,136     << CORRECT (offset seems low, but at least it  
> isn't zero)
> ld $2,0($2)
> bne $2,$0,.L61
> .stabn 68,0,212,.LM111
> .LM111:
> dla $2,MM_RTLinker
> daddiu $3,$2,128     << CORRECT
> li $2,500   # 0x1f4
> sd $2,0($3)
> .stabn 68,0,213,.LM112
> .LM112:
> li $4,4000   # 0xfa0
> jal malloc
> sd $2,0($fp)
> dla $2,MM_RTLinker
> daddiu $3,$2,136      << CORRECT
> ld $2,0($fp)
> sd $2,0($3)
> .stabn 68,0,214,.LM113
> .LM113:
> dla $2,MM_RTLinker
> daddiu $2,$2,136      << CORRECT
> ld $2,0($2)
> bne $2,$0,.L65
> jal abort
> b .L65
> .L61:
>
> I still dump core with this change, but I get much further.
> I have not tested this change at all otherwise -- i.e. on working  
> platforms.
>
> I have not debugged through the gcc/m3cg code to see why
> the offsets are only sometimes lost, or why other platforms
> have no problem. Is MIPS's "global pointer" unusual these days?
> I know SPARC has similar, and all the non-x86, non-AMD64 NT platforms
> had similar, but that's essentially nothing now (I'm not counting  
> IA64).
> I know Linux has PLT (procedure linkage table) and GOT (global  
> offset table).
> Same thing?
> I haven't looked much at other MIPS ports yet.
>
> http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8
>
> is what made it look about like it does -- in particularing using  
> BIT_FIELD_REF.
>
> 1.7:
>
> static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */
>     tree v;
>     int o;
>     tree src_t, dest_t;
>     m3_type src_T, dest_T;
> {
>  if (o == 0 && TREE_TYPE (v) == src_t) {
>    EXPR_PUSH (v);
>  } else {
>    tree adr = m3_build1 (ADDR_EXPR, t_addr, v);
>    if (o != 0) {
>      adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o /  
> BITS_PER_UNIT));
>    }
>    EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t,
>                          m3_cast (build_pointer_type (src_t), adr)));
>  }
> #if 1
>  if (src_T != dest_T) {
>    EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1));
>  }
>  if (debug_vars) {
>    const char *name = "noname";
>    if (v != 0 && DECL_NAME(v) != 0) {
>      name = IDENTIFIER_POINTER(DECL_NAME(v));
>    }
>    fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n",  
> name,
>            o, src_T, dest_T);
>  }
> #else
>  if (src_T != dest_T) {
>    if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) {
>      EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1));
>    } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) {
>      EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1));
>    } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) {
>      /* okay */
>    } else {
>      fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n",
>                  src_T, dest_T);
>    }
>  }
> #endif
> }
> static void
> m3_store (v, o, src_t, dest_t)  /* GCC32OK */
>     tree v;
>     int o;
>     tree src_t, dest_t;
> {
>  tree lhs, rhs;
>  if (TREE_TYPE (EXPR_REF (-1)) == src_t) {
>    rhs = EXPR_REF (-1);
>  } else {
>    rhs = m3_cast (src_t, EXPR_REF (-1));
>  }
>  if (o == 0 && TREE_TYPE (v) == dest_t) {
>    lhs = v;
>  } else {
>    tree f = make_node (FIELD_DECL);
>    TREE_TYPE (f) = dest_t;
>    DECL_ALIGN (f) = TYPE_ALIGN (dest_t);
>    DECL_SIZE (f) = TYPE_SIZE (dest_t);
>    DECL_MODE (f) = TYPE_MODE (dest_t);
>    DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT);
>    DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT);
>    DECL_FIELD_CONTEXT (f) = TREE_TYPE (v);
>    lhs = m3_build2 (COMPONENT_REF, dest_t, v, f);
>  }
> #if 1
>    expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0,  
> 0);
> #else
>  if (dest_t ==  NULL) {
>    expand_assignment (lhs, rhs, 0, 0);
>  } else {
>    expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0,  
> 0);
>  }
> #endif
>  EXPR_POP ();
> }
>
> 1.8:
> static void
> m3_load (tree v, int o,
>  tree src_t, m3_type src_T,
>  tree dst_t, m3_type dst_T) /* GCC32OK */
> {
>  if (o == 0 && TREE_TYPE (v) == src_t) {
>    EXPR_PUSH (v);
>  } else {
>    EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),
>     bitsize_int (o)));
>  }
>  if (src_T != dst_T) {
>    EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1));
>  }
>  if (option_vars_trace) {
>    const char *name = "noname";
>    if (v != 0 && DECL_NAME(v) != 0) {
>      name = IDENTIFIER_POINTER(DECL_NAME(v));
>    }
>    fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n",  
> name,
>     o, src_T, dst_T);
>  }
> }
> static void
> m3_store (tree v, int o, tree src_t, tree dst_t)  /* GCC32OK */
> {
>  tree lhs, rhs;
>  if (TREE_TYPE (EXPR_REF (-1)) == src_t) {
>    rhs = EXPR_REF (-1);
>  } else {
>    rhs = m3_cast (src_t, EXPR_REF (-1));
>  }
>  if (o == 0 && TREE_TYPE (v) == dst_t) {
>    lhs = v;
>  } else {
>    lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),
>       bitsize_int (o));
>  }
>  if (src_t != dst_t) {
>    rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs);
>  }
>  expand_assignment (lhs, rhs, 0, 0);
>  EXPR_POP ();
> }
>
> - Jay

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081108/4bd834d6/attachment-0001.html>

From jay.krell at cornell.edu  Sun Nov  9 02:25:32 2008
From: jay.krell at cornell.edu (Jay)
Date: Sun, 9 Nov 2008 01:25:32 +0000
Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
In-Reply-To: <C47F6B5A-F154-4485-B839-96185B59E3B6@cs.purdue.edu>
References: <COL101-W3DB0BA622C4D31A82B602E6180@phx.gbl>
	<C47F6B5A-F154-4485-B839-96185B59E3B6@cs.purdue.edu>
Message-ID: <COL101-W2CFF8666D1486BD514159E61B0@phx.gbl>


Thanks for the tip/direction.
 
Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the same" but for two systematic diffs:
 
 - AMD64_LINUX has a bunch of added calls to check_nil.
   This is a "bug" in it, inoptimal but ok code, because "first readable address" is 0. It should probably be 4k. I should write a program that iterates starting at 0 to find the value.
  (or better yet, cm3 should have a -configure option that does this? -- for configuration values for which there is a safe default; in general, imho, the targets should be auto configured via C, assuming there is a native development environment available).
 
 - some integers are different (again, systematically) 
 
AMD64_LINUX:
 init_int  656 13 Int.64 init_chars  664 "    typecode " init_int  680 2 Int.64 init_var  688 v.1 0 init_int  696 13 Int.64 init_chars  704 "    typeid   " init_int  720 2 Int.64 init_var  728 v.1 0 init_int  736 13 Int.64 init_chars  744 "RunMainBody: " init_int  760 2 Int.64 init_var  768 v.1 0 init_int  776 19 Int.64 init_chars  784 "RunMainBody: exec: " init_int  808 2 Int.64 init_var  816 v.1 0 init_int  824 17 Int.64 init_chars  832 "ExpandInitStack: "
MIPS64_OPENBSD:
 
 init_chars  664 "    typecode " init_int  680 8796093022208 Int.64 init_var  688 v.1 0 init_int  696 13 Int.64 init_chars  704 "    typeid   " init_int  720 8796093022208 Int.64 init_var  728 v.1 0 init_int  736 13 Int.64 init_chars  744 "RunMainBody: " init_int  760 8796093022208 Int.64 init_var  768 v.1 0 init_int  776 19 Int.64 init_chars  784 "RunMainBody: exec: " init_int  808 8796093022208 Int.64 init_var  816 v.1 0This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where little endian has 2.
 
I don't know what these numbers represent.
I should look at other systems -- preferably a working 64 bit big endian system, if there are any.
 
Could be flags/type with text literals?
 
(Btw, I always leave the TextLiteral.i3 change on my system, lowering the maximum literal length to around 4gig even on 64 bit targets, so 32 bit can bootstrap 64 bit...need to fix the compiler here to do target arithmetic instead of host arithmetic...)
 
Thanks, - Jay



From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sat, 8 Nov 2008 22:55:54 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Have you checked to make sure that the pre-gcc IR makes sense?  Does it see these values as 32-bit or 64-bit?  The reason I ask is that there may be something wrong in your m3middle target specs that cause bad IR.  Once we know the IR looks right then we can try and fix the problem.



Antony Hosking | Associate Professor | Computer Science | Purdue University
305 N. University Street | West Lafayette | IN 47907 | USA
Office +1 765 494 6001 | Mobile +1 765 427 5484


On 8 Nov 2008, at 15:45, Jay wrote:

truncated as usual...along with losing a lot of whitespace..Let's see...Tony, how about switching these #if's?I explain some of why below.static voidm3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){ tree val;#if 0 if (o != 0 || TREE_TYPE (v) != dst_t) {   v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),    bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) {   v = m3_build1 (ADDR_EXPR, t_addr, v);   v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));   v = m3_build1 (INDIRECT_REF, dst_t,    m3_cast (build_pointer_type (dst_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) {   val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP ();}m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){#if 0 if (o != 0 || TREE_TYPE (v) != src_t) {   v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),    bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) {   v = m3_build1 (ADDR_EXPR, t_addr, v);   v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));   v = m3_build1 (INDIRECT_REF, src_t,    m3_cast (build_pointer_type (src_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) {   v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v);}I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).In an early version it seems that many loads/stores of global variables is wrong.The "first" code affected is RTLinker_ExpandModuleTable.PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS;  n_bytes: INTEGER; BEGIN   IF (modules = NIL) THEN     (* first time... *)     max_modules := InitialTableSize;     modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr));     IF (modules = NIL) THEN Cstdlib.abort (); END;   ELSE     n_bytes := max_modules * BYTESIZE (RT0.ModulePtr);     new_mods := Cstdlib.malloc (n_bytes + n_bytes);     IF (new_mods = NIL) THEN Cstdlib.abort (); END;     EVAL Cstring.memcpy (new_mods, modules, n_bytes);     Cstdlib.free (modules);     modules := new_mods;     INC (max_modules, max_modules);   END; END ExpandModuleTable;It should read back modules as NIL.What happens is it reads back modules as not NIL,and then max_modules should also be zero, but is "large", leading to malloc to fail,and abort called.The output of cm3cg -y is reasonable.It indicates offset loads.What is incorrect is that in many loads and stores, but not all, theoffsets are lost.(923) begin_procedure procedure RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable(924) set_source_line source line  208(925) load_nil(926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb(927) set_source_line source line  207(928) set_source_line source line  210(929) load_nil(930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xbHere is the incorrect code:.globl RTLinker__ExpandModuleTable.stabd 46,0,0.stabn 68,0,206,.LM108.LM108:.LFBB8:.set nomips16.ent RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable:.frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerld $2,0($2)      << WRONGbne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $3,MM_RTLinkerld $2,128($3)andi $2,$2,0x0ori $2,$2,0x1f4sd $2,128($3).stabn 68,0,213,.LM112.LM112:li $4,4000   # 0xfa0jal mallocsd $2,0($fp)ld $4,0($fp)dla $3,MM_RTLinkerld $2,136($3)andi $2,$2,0x0or $2,$4,$2sd $2,136($3)     << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerld $2,0($2)       << WRONG bne $2,$0,.L65jal abortb .L65.L61:.stabn 68,0,216,.LM114.LM114:dla $2,MM_RTLinkerld $2,0($2)        << WRONG dsll $2,$2,3sd $2,8($fp).stabn 68,0,217,.LM115.LM115:ld $3,8($fp)ld $2,8($fp)daddu $2,$3,$2move $4,$2jal mallocsd $2,0($fp)ld $2,0($fp)sd $2,16($fp).stabn 68,0,218,.LM116.LM116:ld $2,16($fp)bne $2,$0,.L64jal abort.L64:.stabn 68,0,219,.LM117.LM117:ld $4,16($fp)dla $2,MM_RTLinkerld $2,0($2)          << WRONGld $3,8($fp)move $5,$2move $6,$3jal memcpysd $2,0($fp).stabn 68,0,220,.LM118.LM118:dla $2,MM_RTLinkerld $2,0($2)           << WRONGmove $4,$2jal free.stabn 68,0,221,.LM119.LM119:ld $4,16($fp)dla $3,MM_RTLinkerld $2,136($3)         << OKandi $2,$2,0x0or $2,$4,$2sd $2,136($3)         << OK .stabn 68,0,222,.LM120.LM120:dla $2,MM_RTLinkerld $3,0($2)           << WRONGdla $2,MM_RTLinkerld $2,0($2)           << WRONGdaddu $4,$3,$2dla $3,MM_RTLinkerld $2,128($3)         << OKandi $2,$2,0x0or $2,$2,$4sd $2,128($3)         << OK.L65:.LBE9:.stabn 68,0,224,.LM121.LM121:move $sp,$fpld $31,56($sp)ld $fp,48($sp)ld $28,40($sp)daddiu $sp,$sp,64j $31.end RTLinker__ExpandModuleTableHere is corrected code:RTLinker__ExpandModuleTable:.frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerdaddiu $2,$2,136     << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2)bne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $2,MM_RTLinkerdaddiu $3,$2,128     << CORRECT li $2,500   # 0x1f4sd $2,0($3).stabn 68,0,213,.LM112.LM112:li $4,4000   # 0xfa0jal mallocsd $2,0($fp)dla $2,MM_RTLinkerdaddiu $3,$2,136      << CORRECT ld $2,0($fp)sd $2,0($3).stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerdaddiu $2,$2,136      << CORRECT ld $2,0($2)bne $2,$0,.L65jal abortb .L65.L61:I still dump core with this change, but I get much further.I have not tested this change at all otherwise -- i.e. on working platforms.I have not debugged through the gcc/m3cg code to see whythe offsets are only sometimes lost, or why other platformshave no problem. Is MIPS's "global pointer" unusual these days?I know SPARC has similar, and all the non-x86, non-AMD64 NT platformshad similar, but that's essentially nothing now (I'm not counting IA64).I know Linux has PLT (procedure linkage table) and GOT (global offset table).Same thing?I haven't looked much at other MIPS ports yet.http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8is what made it look about like it does -- in particularing using BIT_FIELD_REF.1.7:static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */    tree v;    int o;    tree src_t, dest_t;    m3_type src_T, dest_T;{ if (o == 0 && TREE_TYPE (v) == src_t) {   EXPR_PUSH (v); } else {   tree adr = m3_build1 (ADDR_EXPR, t_addr, v);   if (o != 0) {     adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT));   }   EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t,                         m3_cast (build_pointer_type (src_t), adr))); }#if 1 if (src_T != dest_T) {   EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) {   const char *name = "noname";   if (v != 0 && DECL_NAME(v) != 0) {     name = IDENTIFIER_POINTER(DECL_NAME(v));   }   fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n", name,           o, src_T, dest_T); }#else if (src_T != dest_T) {   if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) {     EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1));   } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) {     EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1));   } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) {     /* okay */   } else {     fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n",                 src_T, dest_T);   } }#endif}static voidm3_store (v, o, src_t, dest_t)  /* GCC32OK */    tree v;    int o;    tree src_t, dest_t;{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) {   rhs = EXPR_REF (-1); } else {   rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) {   lhs = v; } else {   tree f = make_node (FIELD_DECL);   TREE_TYPE (f) = dest_t;   DECL_ALIGN (f) = TYPE_ALIGN (dest_t);   DECL_SIZE (f) = TYPE_SIZE (dest_t);   DECL_MODE (f) = TYPE_MODE (dest_t);   DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT);   DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT);   DECL_FIELD_CONTEXT (f) = TREE_TYPE (v);   lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); }#if 1   expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);#else if (dest_t ==  NULL) {   expand_assignment (lhs, rhs, 0, 0); } else {   expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); }#endif EXPR_POP ();}1.8:static voidm3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */{ if (o == 0 && TREE_TYPE (v) == src_t) {   EXPR_PUSH (v); } else {   EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),    bitsize_int (o))); } if (src_T != dst_T) {   EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) {   const char *name = "noname";   if (v != 0 && DECL_NAME(v) != 0) {     name = IDENTIFIER_POINTER(DECL_NAME(v));   }   fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n", name,    o, src_T, dst_T); }}static voidm3_store (tree v, int o, tree src_t, tree dst_t)  /* GCC32OK */{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) {   rhs = EXPR_REF (-1); } else {   rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) {   lhs = v; } else {   lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),      bitsize_int (o)); } if (src_t != dst_t) {   rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP ();}- Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081109/2528e9ca/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: AMD64_LINUX_RTLinker.mc
Type: application/octet-stream
Size: 17622 bytes
Desc: not available
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081109/2528e9ca/attachment-0002.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: MIPS64_OPENBSD_RTLinker.mc
Type: application/octet-stream
Size: 17692 bytes
Desc: not available
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081109/2528e9ca/attachment-0003.obj>

From hosking at cs.purdue.edu  Sun Nov  9 11:01:58 2008
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Sun, 9 Nov 2008 10:01:58 +0000
Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
In-Reply-To: <COL101-W2CFF8666D1486BD514159E61B0@phx.gbl>
References: <COL101-W3DB0BA622C4D31A82B602E6180@phx.gbl>
	<C47F6B5A-F154-4485-B839-96185B59E3B6@cs.purdue.edu>
	<COL101-W2CFF8666D1486BD514159E61B0@phx.gbl>
Message-ID: <B9A95C1E-B873-40A9-A045-CD38E32CA5D0@cs.purdue.edu>

Aren't the integer values supposed to be written endian-independently  
in the IR?  I am suspicious about these differences.

On 9 Nov 2008, at 01:25, Jay wrote:

> Thanks for the tip/direction.
>
> Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the  
> same" but for two systematic diffs:
>
>  - AMD64_LINUX has a bunch of added calls to check_nil.
>    This is a "bug" in it, inoptimal but ok code, because "first  
> readable address" is 0. It should probably be 4k. I should write a  
> program that iterates starting at 0 to find the value.
>   (or better yet, cm3 should have a -configure option that does  
> this? -- for configuration values for which there is a safe default;  
> in general, imho, the targets should be auto configured via C,  
> assuming there is a native development environment available).
>
>  - some integers are different (again, systematically)
>
> AMD64_LINUX:
>  init_int  656 13 Int.64
>  init_chars  664 "    typecode "
>  init_int  680 2 Int.64
>  init_var  688 v.1 0
>  init_int  696 13 Int.64
>  init_chars  704 "    typeid   "
>  init_int  720 2 Int.64
>  init_var  728 v.1 0
>  init_int  736 13 Int.64
>  init_chars  744 "RunMainBody: "
>  init_int  760 2 Int.64
>  init_var  768 v.1 0
>  init_int  776 19 Int.64
>  init_chars  784 "RunMainBody: exec: "
>  init_int  808 2 Int.64
>  init_var  816 v.1 0
>  init_int  824 17 Int.64
>  init_chars  832 "ExpandInitStack: "
>
> MIPS64_OPENBSD:
>
>  init_chars  664 "    typecode "
>  init_int  680 8796093022208 Int.64
>  init_var  688 v.1 0
>  init_int  696 13 Int.64
>  init_chars  704 "    typeid   "
>  init_int  720 8796093022208 Int.64
>  init_var  728 v.1 0
>  init_int  736 13 Int.64
>  init_chars  744 "RunMainBody: "
>  init_int  760 8796093022208 Int.64
>  init_var  768 v.1 0
>  init_int  776 19 Int.64
>  init_chars  784 "RunMainBody: exec: "
>  init_int  808 8796093022208 Int.64
>  init_var  816 v.1 0
>
> This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where  
> little endian has 2.
>
> I don't know what these numbers represent.
> I should look at other systems -- preferably a working 64 bit big  
> endian system, if there are any.
>
> Could be flags/type with text literals?
>
> (Btw, I always leave the TextLiteral.i3 change on my system,  
> lowering the maximum literal length to around 4gig even on 64 bit  
> targets, so 32 bit can bootstrap 64 bit...need to fix the compiler  
> here to do target arithmetic instead of host arithmetic...)
>
> Thanks,
>  - Jay
>
>
> From: hosking at cs.purdue.edu
> To: jay.krell at cornell.edu
> Date: Sat, 8 Nov 2008 22:55:54 +0000
> CC: m3devel at elegosoft.com
> Subject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
>
> Have you checked to make sure that the pre-gcc IR makes sense?  Does  
> it see these values as 32-bit or 64-bit?  The reason I ask is that  
> there may be something wrong in your m3middle target specs that  
> cause bad IR.  Once we know the IR looks right then we can try and  
> fix the problem.
>
> Antony Hosking | Associate Professor | Computer Science | Purdue  
> University
> 305 N. University Street | West Lafayette | IN 47907 | USA
> Office +1 765 494 6001 | Mobile +1 765 427 5484
>
>
>
> On 8 Nov 2008, at 15:45, Jay wrote:
>
>
> truncated as usual...along with losing a lot of whitespace..
> Let's see...
>
>
> Tony, how about switching these #if's?
> I explain some of why below.
>
> static void
> m3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t,  
> m3_type dst_T)
> {
>  tree val;
> #if 0
>  if (o != 0 || TREE_TYPE (v) != dst_t) {
>    v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),
>     bitsize_int (o));
>  }
> #else
>  /* failsafe, but inefficient */
>  if (o != 0 || TREE_TYPE (v) != dst_t) {
>    v = m3_build1 (ADDR_EXPR, t_addr, v);
>    v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));
>    v = m3_build1 (INDIRECT_REF, dst_t,
>     m3_cast (build_pointer_type (dst_t), v));
>  }
> #endif
>  TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing  
> problems */
>  val = m3_cast (src_t, EXPR_REF (-1));
>  if (src_T != dst_T) {
>    val = m3_build1 (CONVERT_EXPR, dst_t, val);
>  }
>  add_stmt (build2 (MODIFY_EXPR, dst_t, v, val));
>  EXPR_POP ();
> }
> m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t,  
> m3_type dst_T)
> {
> #if 0
>  if (o != 0 || TREE_TYPE (v) != src_t) {
>    v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),
>     bitsize_int (o));
>  }
> #else
>  /* failsafe, but inefficient */
>  if (o != 0 || TREE_TYPE (v) != src_t) {
>    v = m3_build1 (ADDR_EXPR, t_addr, v);
>    v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));
>    v = m3_build1 (INDIRECT_REF, src_t,
>     m3_cast (build_pointer_type (src_t), v));
>  }
> #endif
>  TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing  
> problems */
>  if (src_T != dst_T) {
>    v = m3_build1 (CONVERT_EXPR, dst_t, v);
>  }
>  EXPR_PUSH (v);
> }
>
> I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).
> In an early version it seems that many loads/stores of global  
> variables is wrong.
> The "first" code affected is RTLinker_ExpandModuleTable.
>
> PROCEDURE ExpandModuleTable () =
>  CONST InitialTableSize = 500;
>  VAR new_mods: ADDRESS;  n_bytes: INTEGER;
>  BEGIN
>    IF (modules = NIL) THEN
>      (* first time... *)
>      max_modules := InitialTableSize;
>      modules := Cstdlib.malloc (InitialTableSize * BYTESIZE  
> (RT0.ModulePtr));
>      IF (modules = NIL) THEN Cstdlib.abort (); END;
>    ELSE
>      n_bytes := max_modules * BYTESIZE (RT0.ModulePtr);
>      new_mods := Cstdlib.malloc (n_bytes + n_bytes);
>      IF (new_mods = NIL) THEN Cstdlib.abort (); END;
>      EVAL Cstring.memcpy (new_mods, modules, n_bytes);
>      Cstdlib.free (modules);
>      modules := new_mods;
>      INC (max_modules, max_modules);
>    END;
>  END ExpandModuleTable;
>
> It should read back modules as NIL.
> What happens is it reads back modules as not NIL,
> and then max_modules should also be zero, but is "large", leading to  
> malloc to fail,
> and abort called.
>
> The output of cm3cg -y is reasonable.
> It indicates offset loads.
>
> What is incorrect is that in many loads and stores, but not all, the
> offsets are lost.
>
> (923) begin_procedure
>  procedure RTLinker__ExpandModuleTable
> RTLinker__ExpandModuleTable(924) set_source_line
>  source line  208
> (925) load_nil
> (926) store
>  store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb
> (927) set_source_line
>  source line  207
> (928) set_source_line
>  source line  210
> (929) load_nil
> (930) load
>  m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xb
>
> Here is the incorrect code:
>
> .globl RTLinker__ExpandModuleTable
> .stabd 46,0,0
> .stabn 68,0,206,.LM108
> .LM108:
> .LFBB8:
> .set nomips16
> .ent RTLinker__ExpandModuleTable
> RTLinker__ExpandModuleTable:
> .frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0
> .mask 0xd0000000,-8
> .fmask 0x00000000,0
> daddiu $sp,$sp,-64
> sd $31,56($sp)
> sd $fp,48($sp)
> sd $28,40($sp)
> move $fp,$sp
> lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
> daddu $28,$28,$25
> daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
> .LBB9:
> .stabn 68,0,208,.LM109
> .LM109:
> sd $0,16($fp)
> .stabn 68,0,210,.LM110
> .LM110:
> dla $2,MM_RTLinker
> ld $2,0($2)      << WRONG
> bne $2,$0,.L61
> .stabn 68,0,212,.LM111
> .LM111:
> dla $3,MM_RTLinker
> ld $2,128($3)
> andi $2,$2,0x0
> ori $2,$2,0x1f4
> sd $2,128($3)
> .stabn 68,0,213,.LM112
> .LM112:
> li $4,4000   # 0xfa0
> jal malloc
> sd $2,0($fp)
> ld $4,0($fp)
> dla $3,MM_RTLinker
> ld $2,136($3)
> andi $2,$2,0x0
> or $2,$4,$2
> sd $2,136($3)     << OK, offset seems low, but it agrees with  
> elsewhere
> .stabn 68,0,214,.LM113
> .LM113:
> dla $2,MM_RTLinker
> ld $2,0($2)       << WRONG
> bne $2,$0,.L65
> jal abort
> b .L65
> .L61:
> .stabn 68,0,216,.LM114
> .LM114:
> dla $2,MM_RTLinker
> ld $2,0($2)        << WRONG
> dsll $2,$2,3
> sd $2,8($fp)
> .stabn 68,0,217,.LM115
> .LM115:
> ld $3,8($fp)
> ld $2,8($fp)
> daddu $2,$3,$2
> move $4,$2
> jal malloc
> sd $2,0($fp)
> ld $2,0($fp)
> sd $2,16($fp)
> .stabn 68,0,218,.LM116
> .LM116:
> ld $2,16($fp)
> bne $2,$0,.L64
> jal abort
> .L64:
> .stabn 68,0,219,.LM117
> .LM117:
> ld $4,16($fp)
> dla $2,MM_RTLinker
> ld $2,0($2)          << WRONG
> ld $3,8($fp)
> move $5,$2
> move $6,$3
> jal memcpy
> sd $2,0($fp)
> .stabn 68,0,220,.LM118
> .LM118:
> dla $2,MM_RTLinker
> ld $2,0($2)           << WRONG
> move $4,$2
> jal free
> .stabn 68,0,221,.LM119
> .LM119:
> ld $4,16($fp)
> dla $3,MM_RTLinker
> ld $2,136($3)         << OK
> andi $2,$2,0x0
> or $2,$4,$2
> sd $2,136($3)         << OK
> .stabn 68,0,222,.LM120
> .LM120:
> dla $2,MM_RTLinker
> ld $3,0($2)           << WRONG
> dla $2,MM_RTLinker
> ld $2,0($2)           << WRONG
> daddu $4,$3,$2
> dla $3,MM_RTLinker
> ld $2,128($3)         << OK
> andi $2,$2,0x0
> or $2,$2,$4
> sd $2,128($3)         << OK
> .L65:
> .LBE9:
> .stabn 68,0,224,.LM121
> .LM121:
> move $sp,$fp
> ld $31,56($sp)
> ld $fp,48($sp)
> ld $28,40($sp)
> daddiu $sp,$sp,64
> j $31
> .end RTLinker__ExpandModuleTable
>
> Here is corrected code:
>
> RTLinker__ExpandModuleTable:
> .frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0
> .mask 0xd0000000,-8
> .fmask 0x00000000,0
> daddiu $sp,$sp,-64
> sd $31,56($sp)
> sd $fp,48($sp)
> sd $28,40($sp)
> move $fp,$sp
> lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
> daddu $28,$28,$25
> daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
> .LBB9:
> .stabn 68,0,208,.LM109
> .LM109:
> sd $0,16($fp)
> .stabn 68,0,210,.LM110
> .LM110:
> dla $2,MM_RTLinker
> daddiu $2,$2,136     << CORRECT (offset seems low, but at least it  
> isn't zero)
> ld $2,0($2)
> bne $2,$0,.L61
> .stabn 68,0,212,.LM111
> .LM111:
> dla $2,MM_RTLinker
> daddiu $3,$2,128     << CORRECT
> li $2,500   # 0x1f4
> sd $2,0($3)
> .stabn 68,0,213,.LM112
> .LM112:
> li $4,4000   # 0xfa0
> jal malloc
> sd $2,0($fp)
> dla $2,MM_RTLinker
> daddiu $3,$2,136      << CORRECT
> ld $2,0($fp)
> sd $2,0($3)
> .stabn 68,0,214,.LM113
> .LM113:
> dla $2,MM_RTLinker
> daddiu $2,$2,136      << CORRECT
> ld $2,0($2)
> bne $2,$0,.L65
> jal abort
> b .L65
> .L61:
>
> I still dump core with this change, but I get much further.
> I have not tested this change at all otherwise -- i.e. on working  
> platforms.
>
> I have not debugged through the gcc/m3cg code to see why
> the offsets are only sometimes lost, or why other platforms
> have no problem. Is MIPS's "global pointer" unusual these days?
> I know SPARC has similar, and all the non-x86, non-AMD64 NT platforms
> had similar, but that's essentially nothing now (I'm not counting  
> IA64).
> I know Linux has PLT (procedure linkage table) and GOT (global  
> offset table).
> Same thing?
> I haven't looked much at other MIPS ports yet.
>
> http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8
>
> is what made it look about like it does -- in particularing using  
> BIT_FIELD_REF.
>
> 1.7:
>
> static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */
>     tree v;
>     int o;
>     tree src_t, dest_t;
>     m3_type src_T, dest_T;
> {
>  if (o == 0 && TREE_TYPE (v) == src_t) {
>    EXPR_PUSH (v);
>  } else {
>    tree adr = m3_build1 (ADDR_EXPR, t_addr, v);
>    if (o != 0) {
>      adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o /  
> BITS_PER_UNIT));
>    }
>    EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t,
>                          m3_cast (build_pointer_type (src_t), adr)));
>  }
> #if 1
>  if (src_T != dest_T) {
>    EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1));
>  }
>  if (debug_vars) {
>    const char *name = "noname";
>    if (v != 0 && DECL_NAME(v) != 0) {
>      name = IDENTIFIER_POINTER(DECL_NAME(v));
>    }
>    fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n",  
> name,
>            o, src_T, dest_T);
>  }
> #else
>  if (src_T != dest_T) {
>    if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) {
>      EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1));
>    } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) {
>      EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1));
>    } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) {
>      /* okay */
>    } else {
>      fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n",
>                  src_T, dest_T);
>    }
>  }
> #endif
> }
> static void
> m3_store (v, o, src_t, dest_t)  /* GCC32OK */
>     tree v;
>     int o;
>     tree src_t, dest_t;
> {
>  tree lhs, rhs;
>  if (TREE_TYPE (EXPR_REF (-1)) == src_t) {
>    rhs = EXPR_REF (-1);
>  } else {
>    rhs = m3_cast (src_t, EXPR_REF (-1));
>  }
>  if (o == 0 && TREE_TYPE (v) == dest_t) {
>    lhs = v;
>  } else {
>    tree f = make_node (FIELD_DECL);
>    TREE_TYPE (f) = dest_t;
>    DECL_ALIGN (f) = TYPE_ALIGN (dest_t);
>    DECL_SIZE (f) = TYPE_SIZE (dest_t);
>    DECL_MODE (f) = TYPE_MODE (dest_t);
>    DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT);
>    DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT);
>    DECL_FIELD_CONTEXT (f) = TREE_TYPE (v);
>    lhs = m3_build2 (COMPONENT_REF, dest_t, v, f);
>  }
> #if 1
>    expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0,  
> 0);
> #else
>  if (dest_t ==  NULL) {
>    expand_assignment (lhs, rhs, 0, 0);
>  } else {
>    expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0,  
> 0);
>  }
> #endif
>  EXPR_POP ();
> }
>
> 1.8:
> static void
> m3_load (tree v, int o,
>  tree src_t, m3_type src_T,
>  tree dst_t, m3_type dst_T) /* GCC32OK */
> {
>  if (o == 0 && TREE_TYPE (v) == src_t) {
>    EXPR_PUSH (v);
>  } else {
>    EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),
>     bitsize_int (o)));
>  }
>  if (src_T != dst_T) {
>    EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1));
>  }
>  if (option_vars_trace) {
>    const char *name = "noname";
>    if (v != 0 && DECL_NAME(v) != 0) {
>      name = IDENTIFIER_POINTER(DECL_NAME(v));
>    }
>    fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n",  
> name,
>     o, src_T, dst_T);
>  }
> }
> static void
> m3_store (tree v, int o, tree src_t, tree dst_t)  /* GCC32OK */
> {
>  tree lhs, rhs;
>  if (TREE_TYPE (EXPR_REF (-1)) == src_t) {
>    rhs = EXPR_REF (-1);
>  } else {
>    rhs = m3_cast (src_t, EXPR_REF (-1));
>  }
>  if (o == 0 && TREE_TYPE (v) == dst_t) {
>    lhs = v;
>  } else {
>    lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),
>       bitsize_int (o));
>  }
>  if (src_t != dst_t) {
>    rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs);
>  }
>  expand_assignment (lhs, rhs, 0, 0);
>  EXPR_POP ();
> }
>
> - Jay
>
> <AMD64_LINUX_RTLinker.mc><MIPS64_OPENBSD_RTLinker.mc>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081109/b8730591/attachment-0001.html>

From jay.krell at cornell.edu  Sun Nov  9 12:10:54 2008
From: jay.krell at cornell.edu (Jay)
Date: Sun, 9 Nov 2008 11:10:54 +0000
Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
In-Reply-To: <B9A95C1E-B873-40A9-A045-CD38E32CA5D0@cs.purdue.edu>
References: <COL101-W3DB0BA622C4D31A82B602E6180@phx.gbl>
	<C47F6B5A-F154-4485-B839-96185B59E3B6@cs.purdue.edu>
	<COL101-W2CFF8666D1486BD514159E61B0@phx.gbl> 
	<B9A95C1E-B873-40A9-A045-CD38E32CA5D0@cs.purdue.edu>
Message-ID: <BAY123-W4494232D2A947D3041E17AE61B0@phx.gbl>


I agree. I am suspicious too. I will investigate more. Thanks.
 
 - Jay



From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 9 Nov 2008 10:01:58 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
Aren't the integer values supposed to be written endian-independently in the IR?  I am suspicious about these differences.




On 9 Nov 2008, at 01:25, Jay wrote:

Thanks for the tip/direction. Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the same" but for two systematic diffs:  - AMD64_LINUX has a bunch of added calls to check_nil.   This is a "bug" in it, inoptimal but ok code, because "first readable address" is 0. It should probably be 4k. I should write a program that iterates starting at 0 to find the value.  (or better yet, cm3 should have a -configure option that does this? -- for configuration values for which there is a safe default; in general, imho, the targets should be auto configured via C, assuming there is a native development environment available).  - some integers are different (again, systematically)  AMD64_LINUX: init_int  656 13 Int.64 init_chars  664 "    typecode " init_int  680 2 Int.64 init_var  688 v.1 0 init_int  696 13 Int.64 init_chars  704 "    typeid   " init_int  720 2 Int.64 init_var  728 v.1 0 init_int  736 13 Int.64 init_chars  744 "RunMainBody: " init_int  760 2 Int.64 init_var  768 v.1 0 init_int  776 19 Int.64 init_chars  784 "RunMainBody: exec: " init_int  808 2 Int.64 init_var  816 v.1 0 init_int  824 17 Int.64 init_chars  832 "ExpandInitStack: "MIPS64_OPENBSD:  init_chars  664 "    typecode " init_int  680 8796093022208 Int.64 init_var  688 v.1 0 init_int  696 13 Int.64 init_chars  704 "    typeid   " init_int  720 8796093022208 Int.64 init_var  728 v.1 0 init_int  736 13 Int.64 init_chars  744 "RunMainBody: " init_int  760 8796093022208 Int.64 init_var  768 v.1 0 init_int  776 19 Int.64 init_chars  784 "RunMainBody: exec: " init_int  808 8796093022208 Int.64 init_var  816 v.1 0This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where little endian has 2. I don't know what these numbers represent.I should look at other systems -- preferably a working 64 bit big endian system, if there are any. Could be flags/type with text literals? (Btw, I always leave the TextLiteral.i3 change on my system, lowering the maximum literal length to around 4gig even on 64 bit targets, so 32 bit can bootstrap 64 bit...need to fix the compiler here to do target arithmetic instead of host arithmetic...) Thanks, - Jay

From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sat, 8 Nov 2008 22:55:54 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Have you checked to make sure that the pre-gcc IR makes sense?  Does it see these values as 32-bit or 64-bit?  The reason I ask is that there may be something wrong in your m3middle target specs that cause bad IR.  Once we know the IR looks right then we can try and fix the problem.



Antony Hosking | Associate Professor | Computer Science | Purdue University
305 N. University Street | West Lafayette | IN 47907 | USA
Office +1 765 494 6001 | Mobile +1 765 427 5484


On 8 Nov 2008, at 15:45, Jay wrote:

truncated as usual...along with losing a lot of whitespace..Let's see...Tony, how about switching these #if's?I explain some of why below.static voidm3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){ tree val;#if 0 if (o != 0 || TREE_TYPE (v) != dst_t) {   v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),    bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) {   v = m3_build1 (ADDR_EXPR, t_addr, v);   v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));   v = m3_build1 (INDIRECT_REF, dst_t,    m3_cast (build_pointer_type (dst_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) {   val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP ();}m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){#if 0 if (o != 0 || TREE_TYPE (v) != src_t) {   v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),    bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) {   v = m3_build1 (ADDR_EXPR, t_addr, v);   v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));   v = m3_build1 (INDIRECT_REF, src_t,    m3_cast (build_pointer_type (src_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) {   v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v);}I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).In an early version it seems that many loads/stores of global variables is wrong.The "first" code affected is RTLinker_ExpandModuleTable.PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS;  n_bytes: INTEGER; BEGIN   IF (modules = NIL) THEN     (* first time... *)     max_modules := InitialTableSize;     modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr));     IF (modules = NIL) THEN Cstdlib.abort (); END;   ELSE     n_bytes := max_modules * BYTESIZE (RT0.ModulePtr);     new_mods := Cstdlib.malloc (n_bytes + n_bytes);     IF (new_mods = NIL) THEN Cstdlib.abort (); END;     EVAL Cstring.memcpy (new_mods, modules, n_bytes);     Cstdlib.free (modules);     modules := new_mods;     INC (max_modules, max_modules);   END; END ExpandModuleTable;It should read back modules as NIL.What happens is it reads back modules as not NIL,and then max_modules should also be zero, but is "large", leading to malloc to fail,and abort called.The output of cm3cg -y is reasonable.It indicates offset loads.What is incorrect is that in many loads and stores, but not all, theoffsets are lost.(923) begin_procedure procedure RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable(924) set_source_line source line  208(925) load_nil(926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb(927) set_source_line source line  207(928) set_source_line source line  210(929) load_nil(930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xbHere is the incorrect code:.globl RTLinker__ExpandModuleTable.stabd 46,0,0.stabn 68,0,206,.LM108.LM108:.LFBB8:.set nomips16.ent RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable:.frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerld $2,0($2)      << WRONGbne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $3,MM_RTLinkerld $2,128($3)andi $2,$2,0x0ori $2,$2,0x1f4sd $2,128($3).stabn 68,0,213,.LM112.LM112:li $4,4000   # 0xfa0jal mallocsd $2,0($fp)ld $4,0($fp)dla $3,MM_RTLinkerld $2,136($3)andi $2,$2,0x0or $2,$4,$2sd $2,136($3)     << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerld $2,0($2)       << WRONG bne $2,$0,.L65jal abortb .L65.L61:.stabn 68,0,216,.LM114.LM114:dla $2,MM_RTLinkerld $2,0($2)        << WRONG dsll $2,$2,3sd $2,8($fp).stabn 68,0,217,.LM115.LM115:ld $3,8($fp)ld $2,8($fp)daddu $2,$3,$2move $4,$2jal mallocsd $2,0($fp)ld $2,0($fp)sd $2,16($fp).stabn 68,0,218,.LM116.LM116:ld $2,16($fp)bne $2,$0,.L64jal abort.L64:.stabn 68,0,219,.LM117.LM117:ld $4,16($fp)dla $2,MM_RTLinkerld $2,0($2)          << WRONGld $3,8($fp)move $5,$2move $6,$3jal memcpysd $2,0($fp).stabn 68,0,220,.LM118.LM118:dla $2,MM_RTLinkerld $2,0($2)           << WRONGmove $4,$2jal free.stabn 68,0,221,.LM119.LM119:ld $4,16($fp)dla $3,MM_RTLinkerld $2,136($3)         << OKandi $2,$2,0x0or $2,$4,$2sd $2,136($3)         << OK .stabn 68,0,222,.LM120.LM120:dla $2,MM_RTLinkerld $3,0($2)           << WRONGdla $2,MM_RTLinkerld $2,0($2)           << WRONGdaddu $4,$3,$2dla $3,MM_RTLinkerld $2,128($3)         << OKandi $2,$2,0x0or $2,$2,$4sd $2,128($3)         << OK.L65:.LBE9:.stabn 68,0,224,.LM121.LM121:move $sp,$fpld $31,56($sp)ld $fp,48($sp)ld $28,40($sp)daddiu $sp,$sp,64j $31.end RTLinker__ExpandModuleTableHere is corrected code:RTLinker__ExpandModuleTable:.frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerdaddiu $2,$2,136     << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2)bne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $2,MM_RTLinkerdaddiu $3,$2,128     << CORRECT li $2,500   # 0x1f4sd $2,0($3).stabn 68,0,213,.LM112.LM112:li $4,4000   # 0xfa0jal mallocsd $2,0($fp)dla $2,MM_RTLinkerdaddiu $3,$2,136      << CORRECT ld $2,0($fp)sd $2,0($3).stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerdaddiu $2,$2,136      << CORRECT ld $2,0($2)bne $2,$0,.L65jal abortb .L65.L61:I still dump core with this change, but I get much further.I have not tested this change at all otherwise -- i.e. on working platforms.I have not debugged through the gcc/m3cg code to see whythe offsets are only sometimes lost, or why other platformshave no problem. Is MIPS's "global pointer" unusual these days?I know SPARC has similar, and all the non-x86, non-AMD64 NT platformshad similar, but that's essentially nothing now (I'm not counting IA64).I know Linux has PLT (procedure linkage table) and GOT (global offset table).Same thing?I haven't looked much at other MIPS ports yet.http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8is what made it look about like it does -- in particularing using BIT_FIELD_REF.1.7:static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */    tree v;    int o;    tree src_t, dest_t;    m3_type src_T, dest_T;{ if (o == 0 && TREE_TYPE (v) == src_t) {   EXPR_PUSH (v); } else {   tree adr = m3_build1 (ADDR_EXPR, t_addr, v);   if (o != 0) {     adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT));   }   EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t,                         m3_cast (build_pointer_type (src_t), adr))); }#if 1 if (src_T != dest_T) {   EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) {   const char *name = "noname";   if (v != 0 && DECL_NAME(v) != 0) {     name = IDENTIFIER_POINTER(DECL_NAME(v));   }   fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n", name,           o, src_T, dest_T); }#else if (src_T != dest_T) {   if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) {     EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1));   } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) {     EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1));   } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) {     /* okay */   } else {     fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n",                 src_T, dest_T);   } }#endif}static voidm3_store (v, o, src_t, dest_t)  /* GCC32OK */    tree v;    int o;    tree src_t, dest_t;{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) {   rhs = EXPR_REF (-1); } else {   rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) {   lhs = v; } else {   tree f = make_node (FIELD_DECL);   TREE_TYPE (f) = dest_t;   DECL_ALIGN (f) = TYPE_ALIGN (dest_t);   DECL_SIZE (f) = TYPE_SIZE (dest_t);   DECL_MODE (f) = TYPE_MODE (dest_t);   DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT);   DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT);   DECL_FIELD_CONTEXT (f) = TREE_TYPE (v);   lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); }#if 1   expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);#else if (dest_t ==  NULL) {   expand_assignment (lhs, rhs, 0, 0); } else {   expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); }#endif EXPR_POP ();}1.8:static voidm3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */{ if (o == 0 && TREE_TYPE (v) == src_t) {   EXPR_PUSH (v); } else {   EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),    bitsize_int (o))); } if (src_T != dst_T) {   EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) {   const char *name = "noname";   if (v != 0 && DECL_NAME(v) != 0) {     name = IDENTIFIER_POINTER(DECL_NAME(v));   }   fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n", name,    o, src_T, dst_T); }}static voidm3_store (tree v, int o, tree src_t, tree dst_t)  /* GCC32OK */{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) {   rhs = EXPR_REF (-1); } else {   rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) {   lhs = v; } else {   lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),      bitsize_int (o)); } if (src_t != dst_t) {   rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP ();}- Jay<AMD64_LINUX_RTLinker.mc><MIPS64_OPENBSD_RTLinker.mc>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081109/f520ce42/attachment-0001.html>

From jay.krell at cornell.edu  Sun Nov  9 15:09:10 2008
From: jay.krell at cornell.edu (Jay)
Date: Sun, 9 Nov 2008 14:09:10 +0000
Subject: [M3devel] AMD64_LINUX gc/stubgen
Message-ID: <COL101-W14DAC6D23849CBD1BE010AE61B0@phx.gbl>


Tony, any tips on this one? :)
 
jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 -sno RemoteView.T -T.M3IMPTAB @M3nogc
 
always succeeds.
 
jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 -sno RemoteView.T   -T.M3IMPTAB
 
usually fails:
****** runtime error:***    NEW() was unable to allocate more memory.***    file "../src/runtime/common/RTAllocator.m3", line 285***Formatter.m3:
 
CONST ChunkSize = 128; (* Must be a power of 2 *)
PROCEDURE New(wr: Wr.T;  width: CARDINAL:= 75): T RAISES {} =  BEGIN    WITH t = NEW(T) DO      t.wr := wr;      t.width := width;      t.indent := 0;        t.numChars := 0;        (* t.stream := RefStream{NIL, ..}; *)        t.buffer := NEW (ExprBuf, 4 * ChunkSize);  << this is the line that fails
 
Lowering chunk size to 64 seemed to make it succeed more often, but not always.
I think I'll go back to MIPS64_OPENBSD..probably easier to track that problem down.
 
 - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081109/00d5c623/attachment-0001.html>

From hosking at cs.purdue.edu  Sun Nov  9 19:02:41 2008
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Sun, 9 Nov 2008 18:02:41 +0000
Subject: [M3devel] AMD64_LINUX gc/stubgen
In-Reply-To: <COL101-W14DAC6D23849CBD1BE010AE61B0@phx.gbl>
References: <COL101-W14DAC6D23849CBD1BE010AE61B0@phx.gbl>
Message-ID: <97806969-D1C7-4D77-8B54-466088B99937@cs.purdue.edu>

Clearly a problem with GC.  I'm guessing you have something wrong  
(pointer alignment?) in the configuration.  I suspect if you run with  
@M3paranoidgc you'll trip over the problem.

On 9 Nov 2008, at 14:09, Jay wrote:

> Tony, any tips on this one? :)
>
> jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 - 
> sno RemoteView.T -T.M3IMPTAB @M3nogc
>
> always succeeds.
>
> jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 - 
> sno RemoteView.T   -T.M3IMPTAB
>
> usually fails:
>
> ***
> *** runtime error:
> ***    NEW() was unable to allocate more memory.
> ***    file "../src/runtime/common/RTAllocator.m3", line 285
> ***
>
> Formatter.m3:
>
> CONST ChunkSize = 128; (* Must be a power of 2 *)
>
> PROCEDURE New(wr: Wr.T;  width: CARDINAL:= 75): T RAISES {} =
>   BEGIN
>     WITH t = NEW(T) DO
>       t.wr := wr;
>       t.width := width;
>       t.indent := 0;
>
>       t.numChars := 0;
>
>       (* t.stream := RefStream{NIL, ..}; *)
>
>       t.buffer := NEW (ExprBuf, 4 * ChunkSize);  << this is the line  
> that fails
>
> Lowering chunk size to 64 seemed to make it succeed more often, but  
> not always.
> I think I'll go back to MIPS64_OPENBSD..probably easier to track  
> that problem down.
>
>  - Jay

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081109/623733d0/attachment-0001.html>

From jay.krell at cornell.edu  Mon Nov 10 09:28:18 2008
From: jay.krell at cornell.edu (Jay)
Date: Mon, 10 Nov 2008 08:28:18 +0000
Subject: [M3devel] AMD64_LINUX gc/stubgen
In-Reply-To: <97806969-D1C7-4D77-8B54-466088B99937@cs.purdue.edu>
References: <COL101-W14DAC6D23849CBD1BE010AE61B0@phx.gbl>
	<97806969-D1C7-4D77-8B54-466088B99937@cs.purdue.edu>
Message-ID: <COL101-W13EBD249838CF77D46F8D2E61A0@phx.gbl>


@M3paranoidgc always succeeds. I see, reading the code, it only does reads. Nevertheless.
 
Simply programs that just allocate memory forever also succeed.
Such as this:
 
jay at amd64a:/dev2/cm3/t$ cat src/t.m3MODULE t EXPORTS Main;
TYPE T1 = RECORD c : ARRAY [0..10000000] OF INTEGER; END;TYPE T2 = RECORD c : ARRAY [0..4095] OF INTEGER; END;
 
VAR a: REFANY;
 
PROCEDURE F1()=VAR b: REFANY;
BEGIN  WHILE TRUE DO    b := NEW(REF T2);    a := NEW(REF T1);  END;END F1;
 
BEGIN  F1();END t.
 
Darn.
 
 - Jay



From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 9 Nov 2008 18:02:41 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] AMD64_LINUX gc/stubgen
Clearly a problem with GC.  I'm guessing you have something wrong (pointer alignment?) in the configuration.  I suspect if you run with @M3paranoidgc you'll trip over the problem.




On 9 Nov 2008, at 14:09, Jay wrote:

Tony, any tips on this one? :) jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 -sno RemoteView.T -T.M3IMPTAB @M3nogc always succeeds. jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 -sno RemoteView.T   -T.M3IMPTAB usually fails:****** runtime error:***    NEW() was unable to allocate more memory.***    file "../src/runtime/common/RTAllocator.m3", line 285***Formatter.m3: CONST ChunkSize = 128; (* Must be a power of 2 *)PROCEDURE New(wr: Wr.T;  width: CARDINAL:= 75): T RAISES {} =  BEGIN    WITH t = NEW(T) DO      t.wr := wr;      t.width := width;      t.indent := 0;        t.numChars := 0;        (* t.stream := RefStream{NIL, ..}; *)        t.buffer := NEW (ExprBuf, 4 * ChunkSize);  << this is the line that fails Lowering chunk size to 64 seemed to make it succeed more often, but not always.I think I'll go back to MIPS64_OPENBSD..probably easier to track that problem down.  - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081110/da2e695b/attachment-0001.html>

From hosking at cs.purdue.edu  Mon Nov 10 12:13:05 2008
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Mon, 10 Nov 2008 11:13:05 +0000
Subject: [M3devel] AMD64_LINUX gc/stubgen
In-Reply-To: <COL101-W13EBD249838CF77D46F8D2E61A0@phx.gbl>
References: <COL101-W14DAC6D23849CBD1BE010AE61B0@phx.gbl>
	<97806969-D1C7-4D77-8B54-466088B99937@cs.purdue.edu>
	<COL101-W13EBD249838CF77D46F8D2E61A0@phx.gbl>
Message-ID: <831F958E-6D14-4DDD-AEF1-7BF3AAA3BC1B@cs.purdue.edu>

Hmm.  Can you try @M3paranoidgc @M3noincremental @M3nogenerational?

On 10 Nov 2008, at 08:28, Jay wrote:

> @M3paranoidgc always succeeds. I see, reading the code, it only does  
> reads. Nevertheless.
>
> Simply programs that just allocate memory forever also succeed.
> Such as this:
>
> jay at amd64a:/dev2/cm3/t$ cat src/t.m3
> MODULE t EXPORTS Main;
> TYPE T1 = RECORD c : ARRAY [0..10000000] OF INTEGER; END;
> TYPE T2 = RECORD c : ARRAY [0..4095] OF INTEGER; END;
>
> VAR a: REFANY;
>
> PROCEDURE F1()=
> VAR b: REFANY;
> BEGIN
>   WHILE TRUE DO
>     b := NEW(REF T2);
>     a := NEW(REF T1);
>   END;
> END F1;
>
> BEGIN
>   F1();
> END t.
>
> Darn.
>
>  - Jay
>
>
> From: hosking at cs.purdue.edu
> To: jay.krell at cornell.edu
> Date: Sun, 9 Nov 2008 18:02:41 +0000
> CC: m3devel at elegosoft.com
> Subject: Re: [M3devel] AMD64_LINUX gc/stubgen
>
>
> Clearly a problem with GC.  I'm guessing you have something wrong  
> (pointer alignment?) in the configuration.  I suspect if you run  
> with @M3paranoidgc you'll trip over the problem.
>
> On 9 Nov 2008, at 14:09, Jay wrote:
>
> Tony, any tips on this one? :)
>
> jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 - 
> sno RemoteView.T -T.M3IMPTAB @M3nogc
>
> always succeeds.
>
> jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 - 
> sno RemoteView.T   -T.M3IMPTAB
>
> usually fails:
>
> ***
> *** runtime error:
> ***    NEW() was unable to allocate more memory.
> ***    file "../src/runtime/common/RTAllocator.m3", line 285
> ***
>
> Formatter.m3:
>
> CONST ChunkSize = 128; (* Must be a power of 2 *)
>
> PROCEDURE New(wr: Wr.T;  width: CARDINAL:= 75): T RAISES {} =
>   BEGIN
>     WITH t = NEW(T) DO
>       t.wr := wr;
>       t.width := width;
>       t.indent := 0;
>
>       t.numChars := 0;
>
>       (* t.stream := RefStream{NIL, ..}; *)
>
>       t.buffer := NEW (ExprBuf, 4 * ChunkSize);  << this is the line  
> that fails
>
> Lowering chunk size to 64 seemed to make it succeed more often, but  
> not always.
> I think I'll go back to MIPS64_OPENBSD..probably easier to track  
> that problem down.
>
>  - Jay
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081110/d1ad8649/attachment-0001.html>

From jay.krell at cornell.edu  Mon Nov 10 20:35:19 2008
From: jay.krell at cornell.edu (Jay)
Date: Mon, 10 Nov 2008 19:35:19 +0000
Subject: [M3devel] AMD64_LINUX gc/stubgen
In-Reply-To: <831F958E-6D14-4DDD-AEF1-7BF3AAA3BC1B@cs.purdue.edu>
References: <COL101-W14DAC6D23849CBD1BE010AE61B0@phx.gbl>
	<97806969-D1C7-4D77-8B54-466088B99937@cs.purdue.edu>
	<COL101-W13EBD249838CF77D46F8D2E61A0@phx.gbl> 
	<831F958E-6D14-4DDD-AEF1-7BF3AAA3BC1B@cs.purdue.edu>
Message-ID: <COL101-W35EAC45CFEB850BBEEFDEDE61A0@phx.gbl>


It /seems/ like any switch or switch combination makes it stop happening.
Admittedly I haven't run a loop just a few runs by hand for each combo.
 
Maybe have it record the allocation quantities and "play them back"?
 
 - Jay


From: hosking at cs.purdue.edu
To: jay.krell at cornell.edu
Date: Mon, 10 Nov 2008 11:13:05 +0000
CC: m3devel at elegosoft.com
Subject: Re: [M3devel] AMD64_LINUX gc/stubgen




Hmm.  Can you try @M3paranoidgc @M3noincremental @M3nogenerational?


On 10 Nov 2008, at 08:28, Jay wrote:


@M3paranoidgc always succeeds. I see, reading the code, it only does reads. Nevertheless.
 
Simply programs that just allocate memory forever also succeed.
Such as this:
 
jay at amd64a:/dev2/cm3/t$ cat src/t.m3
MODULE t EXPORTS Main;
TYPE T1 = RECORD c : ARRAY [0..10000000] OF INTEGER; END;
TYPE T2 = RECORD c : ARRAY [0..4095] OF INTEGER; END;
 
VAR a: REFANY;
 
PROCEDURE F1()=
VAR b: REFANY;
BEGIN
  WHILE TRUE DO
    b := NEW(REF T2);
    a := NEW(REF T1);
  END;
END F1;
 
BEGIN
  F1();
END t.
 
Darn.
 
 - Jay






From: hosking at cs.purdue.edu
To: jay.krell at cornell.edu
Date: Sun, 9 Nov 2008 18:02:41 +0000
CC: m3devel at elegosoft.com
Subject: Re: [M3devel] AMD64_LINUX gc/stubgen



Clearly a problem with GC.  I'm guessing you have something wrong (pointer alignment?) in the configuration.  I suspect if you run with @M3paranoidgc you'll trip over the problem.





On 9 Nov 2008, at 14:09, Jay wrote:


Tony, any tips on this one? :)
 
jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 -sno RemoteView.T -T.M3IMPTAB @M3nogc
 
always succeeds.
 
jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 -sno RemoteView.T   -T.M3IMPTAB
 
usually fails:

***
*** runtime error:
***    NEW() was unable to allocate more memory.
***    file "../src/runtime/common/RTAllocator.m3", line 285
***

Formatter.m3:
 
CONST ChunkSize = 128; (* Must be a power of 2 *)

PROCEDURE New(wr: Wr.T;  width: CARDINAL:= 75): T RAISES {} =
  BEGIN
    WITH t = NEW(T) DO
      t.wr := wr;
      t.width := width;
      t.indent := 0;
  
      t.numChars := 0;
  
      (* t.stream := RefStream{NIL, ..}; *)
  
      t.buffer := NEW (ExprBuf, 4 * ChunkSize);  << this is the line that fails
 
Lowering chunk size to 64 seemed to make it succeed more often, but not always.
I think I'll go back to MIPS64_OPENBSD..probably easier to track that problem down.
 
 - Jay

From rcoleburn at scires.com  Tue Nov 11 00:17:31 2008
From: rcoleburn at scires.com (Randy Coleburn)
Date: Mon, 10 Nov 2008 18:17:31 -0500
Subject: [M3devel] [M3commit] CVS Update: cm3
In-Reply-To: <20081109062016.407EF10D5894@birch.elegosoft.com>
References: <20081109062016.407EF10D5894@birch.elegosoft.com>
Message-ID: <491879AF.1E75.00D7.1@scires.com>

Jay:
 
Does this mean we will now need to put a "cm3cg.common" file into our bin folder on Windows?  
 
Just checking before I do any updates from the repository so I know what needs to change in my target .
 
Regards,
Randy

>>> Jay Krell <jkrell at elego.de> 11/9/2008 1:26 AM >>>
CVSROOT:/usr/cvs
Changes by:jkrell at birch.08/11/09 07:20:15

Modified files:
cm3/m3-sys/cminstall/src/config/: NT386.common cm3.cfg 
cm3/m3-sys/cminstall/src/config-no-install/: AMD64_LINUX 
                                             LINUXLIBC6 
                                             PPC32_OPENBSD 
                                             PPC_DARWIN 
                                             PPC_LINUX SOLgnu 
                                             SOLsun 
                                             SPARC32_LINUX 
                                             SPARC64_LINUX 
                                             SPARC64_OPENBSD 
                                             Unix.common 
Added files:
cm3/m3-sys/cminstall/src/config-no-install/: MIPS64_OPENBSD 
                                             cm3cfg.common 

Log message:
begin combining nt386.common and unix.common (into cm3cg.common)
mips64_openbsd support
being migrating config-no-install/solgnu and solsun to Unix.common
(had this sitting around, in intermediate state)
go back to old formating because that is all the old C written m3build could handle
add probing for host/target specific cm3cg (so I can stop copying it around
as I switch targets)


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081110/03748963/attachment-0001.html>

From jay.krell at cornell.edu  Tue Nov 11 02:06:07 2008
From: jay.krell at cornell.edu (Jay)
Date: Tue, 11 Nov 2008 01:06:07 +0000
Subject: [M3devel] [M3commit] CVS Update: cm3
In-Reply-To: <491879AF.1E75.00D7.1@scires.com>
References: <20081109062016.407EF10D5894@birch.elegosoft.com>
	<491879AF.1E75.00D7.1@scires.com>
Message-ID: <COL101-W586BB50F95A3C239AE2386E6150@phx.gbl>


If you use my config files, yes.
Note that there are two of them, one just includes the other.
Take "the other".
This is just because on my system, I think:
\cm3\cm3.cfg -- probes around 
 => \dev2\cm3.2\m3-sys\cminstall\src\config\NT386 
 => \dev2\cm3.2\m3-sys\cminstall\src\config\NT386.common  
 => \dev2\cm3.2\m3-sys\cminstall\src\config\cm3cfg.common 
 => \dev2\cm3.2\m3-sys\cminstall\src\config-no-install\cm3cfg.common 
 
NT386 is the oddball for me, since for any other targets, for me, it goes:
 
 \cm3\cm3.cfg -- probes around 
  => \dev2\cm3.2\m3-sys\cminstall\src\config-no-install\platform 
  => \dev2\cm3.2\m3-sys\cminstall\src\config-no-install\Unix.common 
  => \dev2\cm3.2\m3-sys\cminstall\src\config-no-install\cm3cfg.common 
 
I'd like to move NT386* to config-no-install, but I avoided doing so so far.
I'd like to replace config with config-no-install, but avoid that too.
 
 - Jay


Date: Mon, 10 Nov 2008 18:17:31 -0500From: rcoleburn at scires.comTo: m3devel at elegosoft.comSubject: Re: [M3devel] [M3commit] CVS Update: cm3
Jay:
 
Does this mean we will now need to put a "cm3cg.common" file into our bin folder on Windows?  
 
Just checking before I do any updates from the repository so I know what needs to change in my target .
 
Regards,
Randy>>> Jay Krell <jkrell at elego.de> 11/9/2008 1:26 AM >>>CVSROOT:/usr/cvsChanges by:jkrell at birch.08/11/09 07:20:15Modified files:cm3/m3-sys/cminstall/src/config/: NT386.common cm3.cfg cm3/m3-sys/cminstall/src/config-no-install/: AMD64_LINUX                                              LINUXLIBC6                                              PPC32_OPENBSD                                              PPC_DARWIN                                              PPC_LINUX SOLgnu                                              SOLsun                                              SPARC32_LINUX                                              SPARC64_LINUX                                              SPARC64_OPENBSD                                              Unix.common Added files:cm3/m3-sys/cminstall/src/config-no-install/: MIPS64_OPENBSD                                              cm3cfg.common Log message:begin combining nt386.common and unix.common (into cm3cg.common)mips64_openbsd supportbeing migrating config-no-install/solgnu and solsun to Unix.common(had this sitting around, in intermediate state)go back to old formating because that is all the old C written m3build could handleadd probing for host/target specific cm3cg (so I can stop copying it aroundas I switch targets)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081111/4d459506/attachment-0001.html>

From rodney.bates at wichita.edu  Tue Nov 11 21:39:09 2008
From: rodney.bates at wichita.edu (Rodney M. Bates)
Date: Tue, 11 Nov 2008 14:39:09 -0600
Subject: [M3devel] Still can't bootstrap cm3
Message-ID: <4919ED6D.6000700@wichita.edu>

We still have serious problems with the cm3 system building.
I have again gotten into a big hampster wheel of several different
failures, and still can't rebuild a working compiler.
I have gone through enough steps that I am not just positive
I have remembered it all right, but it was something like this:

I started with ./do-cm3-front.sh, ./do-cm3-front.sh -ship, and
./install-cm3-compiler.sh upgrade.  That left me with a compiler
that always failed with:

------------------------------------------------------------------------------------------------------------

rodney at yellowstone:~/proj/m3/cm3-new/cm3/m3-sys/m3middle$ cm3 -build
--- building in LINUXLIBC6 ---

ignoring ../src/m3overrides

"/usr/local/cm3/pkg/libm3/src/bundleintf/bundle-ov.tmpl", line 4: quake runtime 
error: undefined variable: ROOT

--procedure--  -line-  -file---
_import_template       4  /usr/local/cm3/pkg/libm3/src/bundleintf/bundle-ov.tmpl
_import_template       4  /usr/local/cm3/pkg/libm3/src/bundleintf/bundle-ov.tmpl
import            274  /usr/local/cm3/pkg/libm3/LINUXLIBC6/.M3EXPORTS
include_dir        11 
/home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3middle/src/m3makefile
                     4 
/home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3middle/LINUXLIBC6/m3make.args

Fatal Error: package build failed

-----------------------------------------------------------------------------------------------------------

Then I tried rebuilding using various do-cm3-* scripts, including after
supplying argument realclean (which, I have learned many times in the
past, is essential to avoid all sorts of problems), with the same results.
Somewhere in here, I started getting the bad version stamp failures too.
Eventually, trying to rebootstrap with ./upgrade.sh, and also manually,
a package at a time, I was consistently getting:

--------------------------------------------------------------------------------------------------------

rodney at yellowstone:~/proj/m3/cm3-new/cm3/m3-sys/m3middle$ cm3 -build
--- building in LINUXLIBC6 ---

ignoring ../src/m3overrides

"/usr/local/cm3/pkg/libm3/LINUXLIBC6/.M3EXPORTS", line 4: quake runtime error: 
unable to open 
"/home/rodney/proj/m3/cm3-new/cm3/m3-libs/m3core/LINUXLIBC6/.M3EXPORTS" for reading

--procedure--  -line-  -file---
import_version      --  <builtin>
import              4  /usr/local/cm3/pkg/libm3/LINUXLIBC6/.M3EXPORTS
include_dir        11 
/home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3middle/src/m3makefile
                     4 
/home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3middle/LINUXLIBC6/m3make.args

Fatal Error: package build failed

-----------------------------------------------------------------------------------------------------------

Perhaps it was the realclean that deleted all the .M3EXPORTS files.
../boot_cm3-core.sh LINUXLIBC6 gave this failure:

-----------------------------------------------------------------------------------------------------------

rodney at yellowstone:~/proj/m3/cm3-new/cm3/scripts$ ./boot-cm3-core.sh
please specify a cross compilation target platform
rodney at yellowstone:~/proj/m3/cm3-new/cm3/scripts$ ./boot-cm3-core.sh LINUXLIBC6
/home/rodney/proj/m3/cm3-new/cm3/scripts/pkgmap.sh -c "cm3 -build -override 
-DROOT='/home/rodney/proj/m3/cm3-new/cm3' -DCM3_VERSION_TEXT='d5.7.0' 
-DCM3_VERSION_NUMBER='050700' -DCM3_LAST_CHANGED='2008-03-16' 
-DM3_BOOTSTRAP=TRUE -keep" m3core libm3 sysutils m3middle m3objfile m3linker 
m3back m3front m3quake cm3 m3scanner m3tools m3cgcat m3cggen m3bundle bitvector 
digraph parseparams realgeometry set slisp sortedtableextras table-list tempfiles
=== package /home/rodney/proj/m3/cm3-new/cm3/m3-libs/m3core ===
  +++ cm3 -build -override -DROOT='/home/rodney/proj/m3/cm3-new/cm3' 
-DCM3_VERSION_TEXT='d5.7.0' -DCM3_VERSION_NUMBER='050700' 
-DCM3_LAST_CHANGED='2008-03-16' -DM3_BOOTSTRAP=TRUE -keep +++
--- building in LINUXLIBC6 ---

new source -> compiling RTHooks.i3
/bin/sh: 
/home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3cc/LINUXLIBC6-LINUXLIBC6/cm3cg: not found
   m3_backend => 127
m3cc (aka cm3cg) failed compiling: RTHooks.ic

---------------------------------------------------------------------------------------------------------

Fortunately, an earlier ./upgrade.sh had built a cm3cg (though it had failed
on all the Modula-3 code).  I got around the above by putting in a symbolic
link from cm3/m3-sys/m3cc/LINUXLIBC6-LINUXLIBC6 to cm3/m3-sys/m3cc/LINUXLIBC6,
so it could find cm3cg.

Then I went through the package-at-a-time order Tony had given in an old post
to the m3devel list.  That had other problems.  Running cm3 in the package
directory failed with the undefined ROOT problem, but do-pkg.sh would do the
build.  But do-pkg.sh doesn't appear to do a ship, so I had to do that in
each package with cm3 -ship.

After building and shipping m3 middle and m3front, package cm3 itself wouldn't
link, wanting libraries for m3back, m3objfile, m3quake, sysutils, m3linker, libm3,
and m3core.  I skipped it and m3gc*, which don't exist now, and went through the
rest of the order, which completed without failures building.  But that gave
me a compiler that fails with a new problem I have never seen before:

---------------------------------------------------------------------------------------------------------

rodney at yellowstone:~/proj/m3/cm3-new/cm3/scripts$ ./install-cm3-compiler.sh upgrade
cp_if: /usr/local/cm3/bin/cm3 and /usr/local/cm3/bin/cm3-d5.7.0 identical
cp_if: /usr/local/cm3/bin/cm3cg and /usr/local/cm3/bin/cm3cg-d5.7.0 identical


***
*** runtime error:
***    A compile-time type is missing.
***    file "../src/Utils.m3"
***

Aborted (core dumped)
cp /home/rodney/proj/m3/cm3-new/cm3/m3-sys/cm3/LINUXLIBC6/cm3 
/usr/local/cm3/bin/cm3-
cp /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3cc/LINUXLIBC6/cm3cg 
/usr/local/cm3/bin/cm3cg-
cp /usr/local/cm3/bin/cm3- /usr/local/cm3/bin/cm3
cp /usr/local/cm3/bin/cm3cg- /usr/local/cm3/bin/cm3cg

------------------------------------------------------------------------------------------------------------

Going back to an older cm3 makes that go away and makes the ROOT problem
recur.  Working around that with cm3 -DROOT="/home/rodney/proj/m3/cm3-new/cm3"
get the compiler to run, but it doesn't do anything, because it thinks
nothing needs to be recompiled.  I could delete stuff, but I fear that would
just start me around another iteration of the same thing.

I don't believe I have ever gotten a bootstrap to work.  I think I have always
had to download a recent binary and go from there.

Any help would be greatly appreciated.

Rodney Bates



From jay.krell at cornell.edu  Wed Nov 12 05:04:44 2008
From: jay.krell at cornell.edu (Jay)
Date: Wed, 12 Nov 2008 04:04:44 +0000
Subject: [M3devel] Still can't bootstrap cm3
In-Reply-To: <4919ED6D.6000700@wichita.edu>
References: <4919ED6D.6000700@wichita.edu>
Message-ID: <COL101-W81AEB201A820491B221953E6140@phx.gbl>


Try starting from a recent snapshot, not necessarily a recent release, and running upgrade.sh or upgrade.py.

 
ROOT should be set in the config file, or as you showed, on the command line.
upgrade.sh/upgrade.py will figure it out and set it on the command line.
I'd kind of like the compiler to set it itself, but that is kind of inappropriate.


It's definitely possible that starting from too old of a release won't work.

 
Avoid any script with "boot" in the name.
Those are I believe for bringing up new targets and esp. cross building, not for
updating mature working systems that can build natively.
The dash in LINUXLIBC6-LINUXLIBC6 confirms it.
The dash appears when the host and target are different.
The first platform is the host, the second is the target.

 
 - Jay

From wagner at elegosoft.com  Wed Nov 12 08:18:55 2008
From: wagner at elegosoft.com (Olaf Wagner)
Date: Wed, 12 Nov 2008 08:18:55 +0100
Subject: [M3devel] Still can't bootstrap cm3
In-Reply-To: <4919ED6D.6000700@wichita.edu>
References: <4919ED6D.6000700@wichita.edu>
Message-ID: <20081112081855.alr1p7k4ggw0k4k0@mail.elegosoft.com>

Quoting "Rodney M. Bates" <rodney.bates at wichita.edu>:

> We still have serious problems with the cm3 system building.
> I have again gotten into a big hampster wheel of several different
> failures, and still can't rebuild a working compiler.
> I have gone through enough steps that I am not just positive
> I have remembered it all right, but it was something like this:

Rodney,

it's not easy to understand what has gone wrong at this point.
Usually, if you have a working system, cm3/scripts/upgrade.sh
should be able to bootstrap to a new set of sources. This is tested
every day in the tinderbox release tests, too. As the last builds seem
to have succeeded, I don't think there is a real problem in our
code base currently. You may want to have a look at test_build_current
in cm3/scripts/regression/defs.sh to see what's done in the regression
builds.

I'd suggest you either start again from the last release or
one of the latest snapshots available from the build machine.
I assume you are not building a cross compiler, but just trying
to perform an upgrade.

Daily snapshots are available from
http://www.opencm3.net/snaps/snapshot-index.html

I hope this helps,

Olaf

> I started with ./do-cm3-front.sh, ./do-cm3-front.sh -ship, and
> ./install-cm3-compiler.sh upgrade.  That left me with a compiler
> that always failed with:
>
> ------------------------------------------------------------------------------------------------------------
>
> rodney at yellowstone:~/proj/m3/cm3-new/cm3/m3-sys/m3middle$ cm3 -build
> --- building in LINUXLIBC6 ---
>
> ignoring ../src/m3overrides
>
> "/usr/local/cm3/pkg/libm3/src/bundleintf/bundle-ov.tmpl", line 4: quake
> runtime error: undefined variable: ROOT
>
> --procedure--  -line-  -file---
> _import_template       4
> /usr/local/cm3/pkg/libm3/src/bundleintf/bundle-ov.tmpl
> _import_template       4
> /usr/local/cm3/pkg/libm3/src/bundleintf/bundle-ov.tmpl
> import            274  /usr/local/cm3/pkg/libm3/LINUXLIBC6/.M3EXPORTS
> include_dir        11
> /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3middle/src/m3makefile
>                     4
> /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3middle/LINUXLIBC6/m3make.args
>
> Fatal Error: package build failed
>
> -----------------------------------------------------------------------------------------------------------
>
> Then I tried rebuilding using various do-cm3-* scripts, including after
> supplying argument realclean (which, I have learned many times in the
> past, is essential to avoid all sorts of problems), with the same results.
> Somewhere in here, I started getting the bad version stamp failures too.
> Eventually, trying to rebootstrap with ./upgrade.sh, and also manually,
> a package at a time, I was consistently getting:
>
> --------------------------------------------------------------------------------------------------------
>
> rodney at yellowstone:~/proj/m3/cm3-new/cm3/m3-sys/m3middle$ cm3 -build
> --- building in LINUXLIBC6 ---
>
> ignoring ../src/m3overrides
>
> "/usr/local/cm3/pkg/libm3/LINUXLIBC6/.M3EXPORTS", line 4: quake runtime
> error: unable to open
> "/home/rodney/proj/m3/cm3-new/cm3/m3-libs/m3core/LINUXLIBC6/.M3EXPORTS"
> for reading
>
> --procedure--  -line-  -file---
> import_version      --  <builtin>
> import              4  /usr/local/cm3/pkg/libm3/LINUXLIBC6/.M3EXPORTS
> include_dir        11
> /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3middle/src/m3makefile
>                     4
> /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3middle/LINUXLIBC6/m3make.args
>
> Fatal Error: package build failed
>
> -----------------------------------------------------------------------------------------------------------
>
> Perhaps it was the realclean that deleted all the .M3EXPORTS files.
> ../boot_cm3-core.sh LINUXLIBC6 gave this failure:
>
> -----------------------------------------------------------------------------------------------------------
>
> rodney at yellowstone:~/proj/m3/cm3-new/cm3/scripts$ ./boot-cm3-core.sh
> please specify a cross compilation target platform
> rodney at yellowstone:~/proj/m3/cm3-new/cm3/scripts$ ./boot-cm3-core.sh
> LINUXLIBC6
> /home/rodney/proj/m3/cm3-new/cm3/scripts/pkgmap.sh -c "cm3 -build
> -override -DROOT='/home/rodney/proj/m3/cm3-new/cm3'
> -DCM3_VERSION_TEXT='d5.7.0' -DCM3_VERSION_NUMBER='050700'
> -DCM3_LAST_CHANGED='2008-03-16' -DM3_BOOTSTRAP=TRUE -keep" m3core libm3
> sysutils m3middle m3objfile m3linker m3back m3front m3quake cm3
> m3scanner m3tools m3cgcat m3cggen m3bundle bitvector digraph
> parseparams realgeometry set slisp sortedtableextras table-list
> tempfiles
> === package /home/rodney/proj/m3/cm3-new/cm3/m3-libs/m3core ===
>  +++ cm3 -build -override -DROOT='/home/rodney/proj/m3/cm3-new/cm3'
> -DCM3_VERSION_TEXT='d5.7.0' -DCM3_VERSION_NUMBER='050700'
> -DCM3_LAST_CHANGED='2008-03-16' -DM3_BOOTSTRAP=TRUE -keep +++
> --- building in LINUXLIBC6 ---
>
> new source -> compiling RTHooks.i3
> /bin/sh:
> /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3cc/LINUXLIBC6-LINUXLIBC6/cm3cg: not
> found
>   m3_backend => 127
> m3cc (aka cm3cg) failed compiling: RTHooks.ic
>
> ---------------------------------------------------------------------------------------------------------
>
> Fortunately, an earlier ./upgrade.sh had built a cm3cg (though it had failed
> on all the Modula-3 code).  I got around the above by putting in a symbolic
> link from cm3/m3-sys/m3cc/LINUXLIBC6-LINUXLIBC6 to   
> cm3/m3-sys/m3cc/LINUXLIBC6,
> so it could find cm3cg.
>
> Then I went through the package-at-a-time order Tony had given in an old post
> to the m3devel list.  That had other problems.  Running cm3 in the package
> directory failed with the undefined ROOT problem, but do-pkg.sh would do the
> build.  But do-pkg.sh doesn't appear to do a ship, so I had to do that in
> each package with cm3 -ship.
>
> After building and shipping m3 middle and m3front, package cm3   
> itself wouldn't
> link, wanting libraries for m3back, m3objfile, m3quake, sysutils,
> m3linker, libm3,
> and m3core.  I skipped it and m3gc*, which don't exist now, and went
> through the
> rest of the order, which completed without failures building.  But that gave
> me a compiler that fails with a new problem I have never seen before:
>
> ---------------------------------------------------------------------------------------------------------
>
> rodney at yellowstone:~/proj/m3/cm3-new/cm3/scripts$
> ./install-cm3-compiler.sh upgrade
> cp_if: /usr/local/cm3/bin/cm3 and /usr/local/cm3/bin/cm3-d5.7.0 identical
> cp_if: /usr/local/cm3/bin/cm3cg and /usr/local/cm3/bin/cm3cg-d5.7.0 identical
>
>
> ***
> *** runtime error:
> ***    A compile-time type is missing.
> ***    file "../src/Utils.m3"
> ***
>
> Aborted (core dumped)
> cp /home/rodney/proj/m3/cm3-new/cm3/m3-sys/cm3/LINUXLIBC6/cm3
> /usr/local/cm3/bin/cm3-
> cp /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3cc/LINUXLIBC6/cm3cg
> /usr/local/cm3/bin/cm3cg-
> cp /usr/local/cm3/bin/cm3- /usr/local/cm3/bin/cm3
> cp /usr/local/cm3/bin/cm3cg- /usr/local/cm3/bin/cm3cg
>
> ------------------------------------------------------------------------------------------------------------
>
> Going back to an older cm3 makes that go away and makes the ROOT problem
> recur.  Working around that with cm3   
> -DROOT="/home/rodney/proj/m3/cm3-new/cm3"
> get the compiler to run, but it doesn't do anything, because it thinks
> nothing needs to be recompiled.  I could delete stuff, but I fear that would
> just start me around another iteration of the same thing.
>
> I don't believe I have ever gotten a bootstrap to work.  I think I   
> have always
> had to download a recent binary and go from there.
>
> Any help would be greatly appreciated.
>
> Rodney Bates



-- 
Olaf Wagner -- elego Software Solutions GmbH
                Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany
phone: +49 30 23 45 86 96  mobile: +49 177 2345 869  fax: +49 30 23 45 86 95
    http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin
Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194



From jay.krell at cornell.edu  Wed Nov 12 08:57:31 2008
From: jay.krell at cornell.edu (Jay)
Date: Wed, 12 Nov 2008 07:57:31 +0000
Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
In-Reply-To: <B9A95C1E-B873-40A9-A045-CD38E32CA5D0@cs.purdue.edu>
References: <COL101-W3DB0BA622C4D31A82B602E6180@phx.gbl>
	<C47F6B5A-F154-4485-B839-96185B59E3B6@cs.purdue.edu>
	<COL101-W2CFF8666D1486BD514159E61B0@phx.gbl> 
	<B9A95C1E-B873-40A9-A045-CD38E32CA5D0@cs.purdue.edu> 
Message-ID: <BAY116-W5161CD049D8CA0E7ABAE4AE6140@phx.gbl>


Target.Little_endian is used to drive some target-specificity in the IR.
   I guess, after all, that is why you have to specify it.
 
The IR for RTLinker.m3 is identical for MIPS64_OPENBSD and SPARC64_SOLARIS, once I set them both to unaligned_procedures, which they both need. (Tangent, I ran m3cgcat on the .ms file a few times instead of the .mc file; that makes it crash.)
 
I can't claim SPARC64_SOLARIS is any paragon of working and correct though, by far.
I should also be able to compare with SPARC64_OPENBSD, I forgot about that, and didn't have an m3cg.
I only ran SPARC64_SOLARIS far enough to discover it needed unaligned procedures.
I think I should fork linux-common and solaris-common off of the minimal 29k openbsd-common, rather than worry about all the stuff in solaris-2-x and linux-libc, over 100k each. Then SPARC64_SOLARIS could/would be in better shape probably.
 
MIPS64_OPENBSD cm3 gets further with unaligned_procedures, along with the parse.c fix -- it gets to a bus error in RunMainBody.
Again an alignment problem, though I think maybe memory memory trashing.
The fault occurs here:
 
    desc_offset := init_depth * ADRSIZE (InitDesc);
      desc := init_stack + desc_offset;      init_depth := desc.low_link; << 
  InitDesc = RECORD    module   : RT0.ModulePtr;    low_link : INTEGER;  END;
desc_offset appears to not be a multiple of 4.
 
I will look into that.
 
Long story short -- consider again the parse.c change?
I admit I still haven't debugged it (or rather, the code that consumes the data produced by it).
 
 - Jay



From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduCC: m3devel at elegosoft.comSubject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Sun, 9 Nov 2008 11:10:54 +0000

I agree. I am suspicious too. I will investigate more. Thanks.  - Jay



From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 9 Nov 2008 10:01:58 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
Aren't the integer values supposed to be written endian-independently in the IR?  I am suspicious about these differences.




On 9 Nov 2008, at 01:25, Jay wrote:

Thanks for the tip/direction. Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the same" but for two systematic diffs:  - AMD64_LINUX has a bunch of added calls to check_nil.   This is a "bug" in it, inoptimal but ok code, because "first readable address" is 0. It should probably be 4k. I should write a program that iterates starting at 0 to find the value.  (or better yet, cm3 should have a -configure option that does this? -- for configuration values for which there is a safe default; in general, imho, the targets should be auto configured via C, assuming there is a native development environment available).  - some integers are different (again, systematically)  AMD64_LINUX: init_int  656 13 Int.64 init_chars  664 "    typecode " init_int  680 2 Int.64 init_var  688 v.1 0 init_int  696 13 Int.64 init_chars  704 "    typeid   " init_int  720 2 Int.64 init_var  728 v.1 0 init_int  736 13 Int.64 init_chars  744 "RunMainBody: " init_int  760 2 Int.64 init_var  768 v.1 0 init_int  776 19 Int.64 init_chars  784 "RunMainBody: exec: " init_int  808 2 Int.64 init_var  816 v.1 0 init_int  824 17 Int.64 init_chars  832 "ExpandInitStack: "MIPS64_OPENBSD:  init_chars  664 "    typecode " init_int  680 8796093022208 Int.64 init_var  688 v.1 0 init_int  696 13 Int.64 init_chars  704 "    typeid   " init_int  720 8796093022208 Int.64 init_var  728 v.1 0 init_int  736 13 Int.64 init_chars  744 "RunMainBody: " init_int  760 8796093022208 Int.64 init_var  768 v.1 0 init_int  776 19 Int.64 init_chars  784 "RunMainBody: exec: " init_int  808 8796093022208 Int.64 init_var  816 v.1 0This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where little endian has 2. I don't know what these numbers represent.I should look at other systems -- preferably a working 64 bit big endian system, if there are any. Could be flags/type with text literals? (Btw, I always leave the TextLiteral.i3 change on my system, lowering the maximum literal length to around 4gig even on 64 bit targets, so 32 bit can bootstrap 64 bit...need to fix the compiler here to do target arithmetic instead of host arithmetic...) Thanks, - Jay

From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sat, 8 Nov 2008 22:55:54 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Have you checked to make sure that the pre-gcc IR makes sense?  Does it see these values as 32-bit or 64-bit?  The reason I ask is that there may be something wrong in your m3middle target specs that cause bad IR.  Once we know the IR looks right then we can try and fix the problem.



Antony Hosking | Associate Professor | Computer Science | Purdue University
305 N. University Street | West Lafayette | IN 47907 | USA
Office +1 765 494 6001 | Mobile +1 765 427 5484


On 8 Nov 2008, at 15:45, Jay wrote:

truncated as usual...along with losing a lot of whitespace..Let's see...Tony, how about switching these #if's?I explain some of why below.static voidm3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){ tree val;#if 0 if (o != 0 || TREE_TYPE (v) != dst_t) {   v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),    bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) {   v = m3_build1 (ADDR_EXPR, t_addr, v);   v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));   v = m3_build1 (INDIRECT_REF, dst_t,    m3_cast (build_pointer_type (dst_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) {   val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP ();}m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){#if 0 if (o != 0 || TREE_TYPE (v) != src_t) {   v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),    bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) {   v = m3_build1 (ADDR_EXPR, t_addr, v);   v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));   v = m3_build1 (INDIRECT_REF, src_t,    m3_cast (build_pointer_type (src_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) {   v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v);}I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).In an early version it seems that many loads/stores of global variables is wrong.The "first" code affected is RTLinker_ExpandModuleTable.PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS;  n_bytes: INTEGER; BEGIN   IF (modules = NIL) THEN     (* first time... *)     max_modules := InitialTableSize;     modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr));     IF (modules = NIL) THEN Cstdlib.abort (); END;   ELSE     n_bytes := max_modules * BYTESIZE (RT0.ModulePtr);     new_mods := Cstdlib.malloc (n_bytes + n_bytes);     IF (new_mods = NIL) THEN Cstdlib.abort (); END;     EVAL Cstring.memcpy (new_mods, modules, n_bytes);     Cstdlib.free (modules);     modules := new_mods;     INC (max_modules, max_modules);   END; END ExpandModuleTable;It should read back modules as NIL.What happens is it reads back modules as not NIL,and then max_modules should also be zero, but is "large", leading to malloc to fail,and abort called.The output of cm3cg -y is reasonable.It indicates offset loads.What is incorrect is that in many loads and stores, but not all, theoffsets are lost.(923) begin_procedure procedure RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable(924) set_source_line source line  208(925) load_nil(926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb(927) set_source_line source line  207(928) set_source_line source line  210(929) load_nil(930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xbHere is the incorrect code:.globl RTLinker__ExpandModuleTable.stabd 46,0,0.stabn 68,0,206,.LM108.LM108:.LFBB8:.set nomips16.ent RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable:.frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerld $2,0($2)      << WRONGbne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $3,MM_RTLinkerld $2,128($3)andi $2,$2,0x0ori $2,$2,0x1f4sd $2,128($3).stabn 68,0,213,.LM112.LM112:li $4,4000   # 0xfa0jal mallocsd $2,0($fp)ld $4,0($fp)dla $3,MM_RTLinkerld $2,136($3)andi $2,$2,0x0or $2,$4,$2sd $2,136($3)     << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerld $2,0($2)       << WRONG bne $2,$0,.L65jal abortb .L65.L61:.stabn 68,0,216,.LM114.LM114:dla $2,MM_RTLinkerld $2,0($2)        << WRONG dsll $2,$2,3sd $2,8($fp).stabn 68,0,217,.LM115.LM115:ld $3,8($fp)ld $2,8($fp)daddu $2,$3,$2move $4,$2jal mallocsd $2,0($fp)ld $2,0($fp)sd $2,16($fp).stabn 68,0,218,.LM116.LM116:ld $2,16($fp)bne $2,$0,.L64jal abort.L64:.stabn 68,0,219,.LM117.LM117:ld $4,16($fp)dla $2,MM_RTLinkerld $2,0($2)          << WRONGld $3,8($fp)move $5,$2move $6,$3jal memcpysd $2,0($fp).stabn 68,0,220,.LM118.LM118:dla $2,MM_RTLinkerld $2,0($2)           << WRONGmove $4,$2jal free.stabn 68,0,221,.LM119.LM119:ld $4,16($fp)dla $3,MM_RTLinkerld $2,136($3)         << OKandi $2,$2,0x0or $2,$4,$2sd $2,136($3)         << OK .stabn 68,0,222,.LM120.LM120:dla $2,MM_RTLinkerld $3,0($2)           << WRONGdla $2,MM_RTLinkerld $2,0($2)           << WRONGdaddu $4,$3,$2dla $3,MM_RTLinkerld $2,128($3)         << OKandi $2,$2,0x0or $2,$2,$4sd $2,128($3)         << OK.L65:.LBE9:.stabn 68,0,224,.LM121.LM121:move $sp,$fpld $31,56($sp)ld $fp,48($sp)ld $28,40($sp)daddiu $sp,$sp,64j $31.end RTLinker__ExpandModuleTableHere is corrected code:RTLinker__ExpandModuleTable:.frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerdaddiu $2,$2,136     << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2)bne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $2,MM_RTLinkerdaddiu $3,$2,128     << CORRECT li $2,500   # 0x1f4sd $2,0($3).stabn 68,0,213,.LM112.LM112:li $4,4000   # 0xfa0jal mallocsd $2,0($fp)dla $2,MM_RTLinkerdaddiu $3,$2,136      << CORRECT ld $2,0($fp)sd $2,0($3).stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerdaddiu $2,$2,136      << CORRECT ld $2,0($2)bne $2,$0,.L65jal abortb .L65.L61:I still dump core with this change, but I get much further.I have not tested this change at all otherwise -- i.e. on working platforms.I have not debugged through the gcc/m3cg code to see whythe offsets are only sometimes lost, or why other platformshave no problem. Is MIPS's "global pointer" unusual these days?I know SPARC has similar, and all the non-x86, non-AMD64 NT platformshad similar, but that's essentially nothing now (I'm not counting IA64).I know Linux has PLT (procedure linkage table) and GOT (global offset table).Same thing?I haven't looked much at other MIPS ports yet.http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8is what made it look about like it does -- in particularing using BIT_FIELD_REF.1.7:static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */    tree v;    int o;    tree src_t, dest_t;    m3_type src_T, dest_T;{ if (o == 0 && TREE_TYPE (v) == src_t) {   EXPR_PUSH (v); } else {   tree adr = m3_build1 (ADDR_EXPR, t_addr, v);   if (o != 0) {     adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT));   }   EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t,                         m3_cast (build_pointer_type (src_t), adr))); }#if 1 if (src_T != dest_T) {   EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) {   const char *name = "noname";   if (v != 0 && DECL_NAME(v) != 0) {     name = IDENTIFIER_POINTER(DECL_NAME(v));   }   fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n", name,           o, src_T, dest_T); }#else if (src_T != dest_T) {   if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) {     EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1));   } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) {     EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1));   } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) {     /* okay */   } else {     fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n",                 src_T, dest_T);   } }#endif}static voidm3_store (v, o, src_t, dest_t)  /* GCC32OK */    tree v;    int o;    tree src_t, dest_t;{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) {   rhs = EXPR_REF (-1); } else {   rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) {   lhs = v; } else {   tree f = make_node (FIELD_DECL);   TREE_TYPE (f) = dest_t;   DECL_ALIGN (f) = TYPE_ALIGN (dest_t);   DECL_SIZE (f) = TYPE_SIZE (dest_t);   DECL_MODE (f) = TYPE_MODE (dest_t);   DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT);   DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT);   DECL_FIELD_CONTEXT (f) = TREE_TYPE (v);   lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); }#if 1   expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);#else if (dest_t ==  NULL) {   expand_assignment (lhs, rhs, 0, 0); } else {   expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); }#endif EXPR_POP ();}1.8:static voidm3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */{ if (o == 0 && TREE_TYPE (v) == src_t) {   EXPR_PUSH (v); } else {   EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),    bitsize_int (o))); } if (src_T != dst_T) {   EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) {   const char *name = "noname";   if (v != 0 && DECL_NAME(v) != 0) {     name = IDENTIFIER_POINTER(DECL_NAME(v));   }   fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n", name,    o, src_T, dst_T); }}static voidm3_store (tree v, int o, tree src_t, tree dst_t)  /* GCC32OK */{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) {   rhs = EXPR_REF (-1); } else {   rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) {   lhs = v; } else {   lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),      bitsize_int (o)); } if (src_t != dst_t) {   rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP ();}- Jay<AMD64_LINUX_RTLinker.mc><MIPS64_OPENBSD_RTLinker.mc>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081112/afd2a674/attachment-0001.html>

From jay.krell at cornell.edu  Wed Nov 12 09:05:03 2008
From: jay.krell at cornell.edu (Jay)
Date: Wed, 12 Nov 2008 08:05:03 +0000
Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
In-Reply-To: <BAY116-W5161CD049D8CA0E7ABAE4AE6140@phx.gbl>
References: <COL101-W3DB0BA622C4D31A82B602E6180@phx.gbl>
	<C47F6B5A-F154-4485-B839-96185B59E3B6@cs.purdue.edu>
	<COL101-W2CFF8666D1486BD514159E61B0@phx.gbl> 
	<B9A95C1E-B873-40A9-A045-CD38E32CA5D0@cs.purdue.edu>  
	<BAY116-W5161CD049D8CA0E7ABAE4AE6140@phx.gbl>
Message-ID: <BAY121-W536D2BD8E7F1E500416F5FE6140@phx.gbl>


ps: it takes over 20 seconds for MIPS64_OPENBSD to crash in RunMainBody, which makes me suspect the platform is impractically slow.
I can only hope that optimization and/or not crashing makes it much faster.
The bootstrap build -- assemble a bunch, compile a little C, link, is also slow though.
System is 200MHz+ and 256meg RAM though, ought to suffice..
 
- Jay



From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduDate: Wed, 12 Nov 2008 07:57:31 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??

Target.Little_endian is used to drive some target-specificity in the IR.   I guess, after all, that is why you have to specify it. The IR for RTLinker.m3 is identical for MIPS64_OPENBSD and SPARC64_SOLARIS, once I set them both to unaligned_procedures, which they both need. (Tangent, I ran m3cgcat on the .ms file a few times instead of the .mc file; that makes it crash.) I can't claim SPARC64_SOLARIS is any paragon of working and correct though, by far.I should also be able to compare with SPARC64_OPENBSD, I forgot about that, and didn't have an m3cg.I only ran SPARC64_SOLARIS far enough to discover it needed unaligned procedures.I think I should fork linux-common and solaris-common off of the minimal 29k openbsd-common, rather than worry about all the stuff in solaris-2-x and linux-libc, over 100k each. Then SPARC64_SOLARIS could/would be in better shape probably. MIPS64_OPENBSD cm3 gets further with unaligned_procedures, along with the parse.c fix -- it gets to a bus error in RunMainBody.Again an alignment problem, though I think maybe memory memory trashing.The fault occurs here:     desc_offset := init_depth * ADRSIZE (InitDesc);      desc := init_stack + desc_offset;      init_depth := desc.low_link; <<   InitDesc = RECORD    module   : RT0.ModulePtr;    low_link : INTEGER;  END;desc_offset appears to not be a multiple of 4. I will look into that. Long story short -- consider again the parse.c change?I admit I still haven't debugged it (or rather, the code that consumes the data produced by it).  - Jay



From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduCC: m3devel at elegosoft.comSubject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Sun, 9 Nov 2008 11:10:54 +0000

I agree. I am suspicious too. I will investigate more. Thanks.  - Jay



From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 9 Nov 2008 10:01:58 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
Aren't the integer values supposed to be written endian-independently in the IR?  I am suspicious about these differences.




On 9 Nov 2008, at 01:25, Jay wrote:

Thanks for the tip/direction. Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the same" but for two systematic diffs:  - AMD64_LINUX has a bunch of added calls to check_nil.   This is a "bug" in it, inoptimal but ok code, because "first readable address" is 0. It should probably be 4k. I should write a program that iterates starting at 0 to find the value.  (or better yet, cm3 should have a -configure option that does this? -- for configuration values for which there is a safe default; in general, imho, the targets should be auto configured via C, assuming there is a native development environment available).  - some integers are different (again, systematically)  AMD64_LINUX: init_int  656 13 Int.64 init_chars  664 "    typecode " init_int  680 2 Int.64 init_var  688 v.1 0 init_int  696 13 Int.64 init_chars  704 "    typeid   " init_int  720 2 Int.64 init_var  728 v.1 0 init_int  736 13 Int.64 init_chars  744 "RunMainBody: " init_int  760 2 Int.64 init_var  768 v.1 0 init_int  776 19 Int.64 init_chars  784 "RunMainBody: exec: " init_int  808 2 Int.64 init_var  816 v.1 0 init_int  824 17 Int.64 init_chars  832 "ExpandInitStack: "MIPS64_OPENBSD:  init_chars  664 "    typecode " init_int  680 8796093022208 Int.64 init_var  688 v.1 0 init_int  696 13 Int.64 init_chars  704 "    typeid   " init_int  720 8796093022208 Int.64 init_var  728 v.1 0 init_int  736 13 Int.64 init_chars  744 "RunMainBody: " init_int  760 8796093022208 Int.64 init_var  768 v.1 0 init_int  776 19 Int.64 init_chars  784 "RunMainBody: exec: " init_int  808 8796093022208 Int.64 init_var  816 v.1 0This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where little endian has 2. I don't know what these numbers represent.I should look at other systems -- preferably a working 64 bit big endian system, if there are any. Could be flags/type with text literals? (Btw, I always leave the TextLiteral.i3 change on my system, lowering the maximum literal length to around 4gig even on 64 bit targets, so 32 bit can bootstrap 64 bit...need to fix the compiler here to do target arithmetic instead of host arithmetic...) Thanks, - Jay

From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sat, 8 Nov 2008 22:55:54 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Have you checked to make sure that the pre-gcc IR makes sense?  Does it see these values as 32-bit or 64-bit?  The reason I ask is that there may be something wrong in your m3middle target specs that cause bad IR.  Once we know the IR looks right then we can try and fix the problem.



Antony Hosking | Associate Professor | Computer Science | Purdue University
305 N. University Street | West Lafayette | IN 47907 | USA
Office +1 765 494 6001 | Mobile +1 765 427 5484


On 8 Nov 2008, at 15:45, Jay wrote:

truncated as usual...along with losing a lot of whitespace..Let's see...Tony, how about switching these #if's?I explain some of why below.static voidm3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){ tree val;#if 0 if (o != 0 || TREE_TYPE (v) != dst_t) {   v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),    bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) {   v = m3_build1 (ADDR_EXPR, t_addr, v);   v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));   v = m3_build1 (INDIRECT_REF, dst_t,    m3_cast (build_pointer_type (dst_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) {   val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP ();}m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){#if 0 if (o != 0 || TREE_TYPE (v) != src_t) {   v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),    bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) {   v = m3_build1 (ADDR_EXPR, t_addr, v);   v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));   v = m3_build1 (INDIRECT_REF, src_t,    m3_cast (build_pointer_type (src_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) {   v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v);}I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).In an early version it seems that many loads/stores of global variables is wrong.The "first" code affected is RTLinker_ExpandModuleTable.PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS;  n_bytes: INTEGER; BEGIN   IF (modules = NIL) THEN     (* first time... *)     max_modules := InitialTableSize;     modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr));     IF (modules = NIL) THEN Cstdlib.abort (); END;   ELSE     n_bytes := max_modules * BYTESIZE (RT0.ModulePtr);     new_mods := Cstdlib.malloc (n_bytes + n_bytes);     IF (new_mods = NIL) THEN Cstdlib.abort (); END;     EVAL Cstring.memcpy (new_mods, modules, n_bytes);     Cstdlib.free (modules);     modules := new_mods;     INC (max_modules, max_modules);   END; END ExpandModuleTable;It should read back modules as NIL.What happens is it reads back modules as not NIL,and then max_modules should also be zero, but is "large", leading to malloc to fail,and abort called.The output of cm3cg -y is reasonable.It indicates offset loads.What is incorrect is that in many loads and stores, but not all, theoffsets are lost.(923) begin_procedure procedure RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable(924) set_source_line source line  208(925) load_nil(926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb(927) set_source_line source line  207(928) set_source_line source line  210(929) load_nil(930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xbHere is the incorrect code:.globl RTLinker__ExpandModuleTable.stabd 46,0,0.stabn 68,0,206,.LM108.LM108:.LFBB8:.set nomips16.ent RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable:.frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerld $2,0($2)      << WRONGbne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $3,MM_RTLinkerld $2,128($3)andi $2,$2,0x0ori $2,$2,0x1f4sd $2,128($3).stabn 68,0,213,.LM112.LM112:li $4,4000   # 0xfa0jal mallocsd $2,0($fp)ld $4,0($fp)dla $3,MM_RTLinkerld $2,136($3)andi $2,$2,0x0or $2,$4,$2sd $2,136($3)     << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerld $2,0($2)       << WRONG bne $2,$0,.L65jal abortb .L65.L61:.stabn 68,0,216,.LM114.LM114:dla $2,MM_RTLinkerld $2,0($2)        << WRONG dsll $2,$2,3sd $2,8($fp).stabn 68,0,217,.LM115.LM115:ld $3,8($fp)ld $2,8($fp)daddu $2,$3,$2move $4,$2jal mallocsd $2,0($fp)ld $2,0($fp)sd $2,16($fp).stabn 68,0,218,.LM116.LM116:ld $2,16($fp)bne $2,$0,.L64jal abort.L64:.stabn 68,0,219,.LM117.LM117:ld $4,16($fp)dla $2,MM_RTLinkerld $2,0($2)          << WRONGld $3,8($fp)move $5,$2move $6,$3jal memcpysd $2,0($fp).stabn 68,0,220,.LM118.LM118:dla $2,MM_RTLinkerld $2,0($2)           << WRONGmove $4,$2jal free.stabn 68,0,221,.LM119.LM119:ld $4,16($fp)dla $3,MM_RTLinkerld $2,136($3)         << OKandi $2,$2,0x0or $2,$4,$2sd $2,136($3)         << OK .stabn 68,0,222,.LM120.LM120:dla $2,MM_RTLinkerld $3,0($2)           << WRONGdla $2,MM_RTLinkerld $2,0($2)           << WRONGdaddu $4,$3,$2dla $3,MM_RTLinkerld $2,128($3)         << OKandi $2,$2,0x0or $2,$2,$4sd $2,128($3)         << OK.L65:.LBE9:.stabn 68,0,224,.LM121.LM121:move $sp,$fpld $31,56($sp)ld $fp,48($sp)ld $28,40($sp)daddiu $sp,$sp,64j $31.end RTLinker__ExpandModuleTableHere is corrected code:RTLinker__ExpandModuleTable:.frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerdaddiu $2,$2,136     << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2)bne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $2,MM_RTLinkerdaddiu $3,$2,128     << CORRECT li $2,500   # 0x1f4sd $2,0($3).stabn 68,0,213,.LM112.LM112:li $4,4000   # 0xfa0jal mallocsd $2,0($fp)dla $2,MM_RTLinkerdaddiu $3,$2,136      << CORRECT ld $2,0($fp)sd $2,0($3).stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerdaddiu $2,$2,136      << CORRECT ld $2,0($2)bne $2,$0,.L65jal abortb .L65.L61:I still dump core with this change, but I get much further.I have not tested this change at all otherwise -- i.e. on working platforms.I have not debugged through the gcc/m3cg code to see whythe offsets are only sometimes lost, or why other platformshave no problem. Is MIPS's "global pointer" unusual these days?I know SPARC has similar, and all the non-x86, non-AMD64 NT platformshad similar, but that's essentially nothing now (I'm not counting IA64).I know Linux has PLT (procedure linkage table) and GOT (global offset table).Same thing?I haven't looked much at other MIPS ports yet.http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8is what made it look about like it does -- in particularing using BIT_FIELD_REF.1.7:static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */    tree v;    int o;    tree src_t, dest_t;    m3_type src_T, dest_T;{ if (o == 0 && TREE_TYPE (v) == src_t) {   EXPR_PUSH (v); } else {   tree adr = m3_build1 (ADDR_EXPR, t_addr, v);   if (o != 0) {     adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT));   }   EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t,                         m3_cast (build_pointer_type (src_t), adr))); }#if 1 if (src_T != dest_T) {   EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) {   const char *name = "noname";   if (v != 0 && DECL_NAME(v) != 0) {     name = IDENTIFIER_POINTER(DECL_NAME(v));   }   fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n", name,           o, src_T, dest_T); }#else if (src_T != dest_T) {   if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) {     EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1));   } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) {     EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1));   } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) {     /* okay */   } else {     fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n",                 src_T, dest_T);   } }#endif}static voidm3_store (v, o, src_t, dest_t)  /* GCC32OK */    tree v;    int o;    tree src_t, dest_t;{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) {   rhs = EXPR_REF (-1); } else {   rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) {   lhs = v; } else {   tree f = make_node (FIELD_DECL);   TREE_TYPE (f) = dest_t;   DECL_ALIGN (f) = TYPE_ALIGN (dest_t);   DECL_SIZE (f) = TYPE_SIZE (dest_t);   DECL_MODE (f) = TYPE_MODE (dest_t);   DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT);   DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT);   DECL_FIELD_CONTEXT (f) = TREE_TYPE (v);   lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); }#if 1   expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);#else if (dest_t ==  NULL) {   expand_assignment (lhs, rhs, 0, 0); } else {   expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); }#endif EXPR_POP ();}1.8:static voidm3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */{ if (o == 0 && TREE_TYPE (v) == src_t) {   EXPR_PUSH (v); } else {   EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),    bitsize_int (o))); } if (src_T != dst_T) {   EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) {   const char *name = "noname";   if (v != 0 && DECL_NAME(v) != 0) {     name = IDENTIFIER_POINTER(DECL_NAME(v));   }   fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n", name,    o, src_T, dst_T); }}static voidm3_store (tree v, int o, tree src_t, tree dst_t)  /* GCC32OK */{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) {   rhs = EXPR_REF (-1); } else {   rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) {   lhs = v; } else {   lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),      bitsize_int (o)); } if (src_t != dst_t) {   rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP ();}- Jay<AMD64_LINUX_RTLinker.mc><MIPS64_OPENBSD_RTLinker.mc>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081112/c2cc1a47/attachment-0001.html>

From rcoleburn at scires.com  Wed Nov 12 19:49:19 2008
From: rcoleburn at scires.com (Randy Coleburn)
Date: Wed, 12 Nov 2008 13:49:19 -0500
Subject: [M3devel] [M3commit] CVS Update: cm3
In-Reply-To: <20081112152142.02AEA10D5970@birch.elegosoft.com>
References: <20081112152142.02AEA10D5970@birch.elegosoft.com>
Message-ID: <491ADDBA.1E75.00D7.1@scires.com>

Rodney:
 
Is this change simply to make PM3 and CM3 more compatible wrt pickles, or does it alter current behavior for folks who only use CM3 ? (i.e., will it result in a change to the way CM3 pickles are represented that may affect current CM3 users?)
 
Regards,
Randy

>>> Rodney M. Bates <rodney at elego.de> 11/12/2008 4:21 PM >>>
CVSROOT:/usr/cvs
Changes by:rodney at birch.08/11/12 16:21:41

Modified files:
cm3/m3-libs/libm3/src/pickle/ver2/: Pickle2.m3 

Log message:
Pickles in both PM3 and CM3 have some hard-coded
fingerprints for common types, but the values
are not consistent about byte order.

Fix CM2 pickle-reading code to accept values of types
NULL, ROOT, UNTRACED ROOT, ADDRESS, AND REFANY,
written by PM3 pickle-writing code.  Probably,
the latter two can't even happen in a pickle, unless
somebody wrote a custom special for them.

This still won't handle a more complex type that depends
on these types, e.g., an object that contains a field of
one of them.  This latter is probably not feasable, because
it would require "decrypting" fingerprints, and the hash
algorithm for fingerprints is lossy.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081112/456771f5/attachment-0001.html>

From hosking at cs.purdue.edu  Wed Nov 12 21:46:53 2008
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Wed, 12 Nov 2008 20:46:53 +0000
Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
In-Reply-To: <BAY116-W5161CD049D8CA0E7ABAE4AE6140@phx.gbl>
References: <COL101-W3DB0BA622C4D31A82B602E6180@phx.gbl>
	<C47F6B5A-F154-4485-B839-96185B59E3B6@cs.purdue.edu>
	<COL101-W2CFF8666D1486BD514159E61B0@phx.gbl>
	<B9A95C1E-B873-40A9-A045-CD38E32CA5D0@cs.purdue.edu>
	<BAY116-W5161CD049D8CA0E7ABAE4AE6140@phx.gbl>
Message-ID: <80AE04BD-4819-4085-8EBE-5996E8E6DB93@cs.purdue.edu>

Jay, do you have access to an AMD64_DARWIN box?   I did get things  
working on that 64-bit platform.   Or is it an endianness issue you  
are dealing with.  In any case, I will not have a lot of bandwidth for  
doing much in the way of tackling this unless it becomes critical path  
for me soon.

Antony Hosking | Associate Professor | Computer Science | Purdue  
University
305 N. University Street | West Lafayette | IN 47907 | USA
Office +1 765 494 6001 | Mobile +1 765 427 5484



On 12 Nov 2008, at 07:57, Jay wrote:

> Target.Little_endian is used to drive some target-specificity in the  
> IR.
>    I guess, after all, that is why you have to specify it.
>
> The IR for RTLinker.m3 is identical for MIPS64_OPENBSD and  
> SPARC64_SOLARIS, once I set them both to unaligned_procedures, which  
> they both need. (Tangent, I ran m3cgcat on the .ms file a few times  
> instead of the .mc file; that makes it crash.)
>
> I can't claim SPARC64_SOLARIS is any paragon of working and correct  
> though, by far.
> I should also be able to compare with SPARC64_OPENBSD, I forgot  
> about that, and didn't have an m3cg.
> I only ran SPARC64_SOLARIS far enough to discover it needed  
> unaligned procedures.
> I think I should fork linux-common and solaris-common off of the  
> minimal 29k openbsd-common, rather than worry about all the stuff in  
> solaris-2-x and linux-libc, over 100k each. Then SPARC64_SOLARIS  
> could/would be in better shape probably.
>
> MIPS64_OPENBSD cm3 gets further with unaligned_procedures, along  
> with the parse.c fix -- it gets to a bus error in RunMainBody.
> Again an alignment problem, though I think maybe memory memory  
> trashing.
> The fault occurs here:
>
>     desc_offset := init_depth * ADRSIZE (InitDesc);
>
>       desc := init_stack + desc_offset;
>       init_depth := desc.low_link; <<
>
>   InitDesc = RECORD
>     module   : RT0.ModulePtr;
>     low_link : INTEGER;
>   END;
>
> desc_offset appears to not be a multiple of 4.
>
> I will look into that.
>
> Long story short -- consider again the parse.c change?
> I admit I still haven't debugged it (or rather, the code that  
> consumes the data produced by it).
>
>  - Jay
>
>
>
> From: jay.krell at cornell.edu
> To: hosking at cs.purdue.edu
> CC: m3devel at elegosoft.com
> Subject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
> Date: Sun, 9 Nov 2008 11:10:54 +0000
>
> I agree. I am suspicious too. I will investigate more. Thanks.
>
>  - Jay
>
>
>
>
> From: hosking at cs.purdue.edu
> To: jay.krell at cornell.edu
> Date: Sun, 9 Nov 2008 10:01:58 +0000
> CC: m3devel at elegosoft.com
> Subject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
>
>
>
> Aren't the integer values supposed to be written endian- 
> independently in the IR?  I am suspicious about these differences.
>
> On 9 Nov 2008, at 01:25, Jay wrote:
>
> Thanks for the tip/direction.
>
> Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the  
> same" but for two systematic diffs:
>
>  - AMD64_LINUX has a bunch of added calls to check_nil.
>    This is a "bug" in it, inoptimal but ok code, because "first  
> readable address" is 0. It should probably be 4k. I should write a  
> program that iterates starting at 0 to find the value.
>   (or better yet, cm3 should have a -configure option that does  
> this? -- for configuration values for which there is a safe default;  
> in general, imho, the targets should be auto configured via C,  
> assuming there is a native development environment available).
>
>  - some integers are different (again, systematically)
>
> AMD64_LINUX:
>  init_int  656 13 Int.64
>  init_chars  664 "    typecode "
>  init_int  680 2 Int.64
>  init_var  688 v.1 0
>  init_int  696 13 Int.64
>  init_chars  704 "    typeid   "
>  init_int  720 2 Int.64
>  init_var  728 v.1 0
>  init_int  736 13 Int.64
>  init_chars  744 "RunMainBody: "
>  init_int  760 2 Int.64
>  init_var  768 v.1 0
>  init_int  776 19 Int.64
>  init_chars  784 "RunMainBody: exec: "
>  init_int  808 2 Int.64
>  init_var  816 v.1 0
>  init_int  824 17 Int.64
>  init_chars  832 "ExpandInitStack: "
>
> MIPS64_OPENBSD:
>
>  init_chars  664 "    typecode "
>  init_int  680 8796093022208 Int.64
>  init_var  688 v.1 0
>  init_int  696 13 Int.64
>  init_chars  704 "    typeid   "
>  init_int  720 8796093022208 Int.64
>  init_var  728 v.1 0
>  init_int  736 13 Int.64
>  init_chars  744 "RunMainBody: "
>  init_int  760 8796093022208 Int.64
>  init_var  768 v.1 0
>  init_int  776 19 Int.64
>  init_chars  784 "RunMainBody: exec: "
>  init_int  808 8796093022208 Int.64
>  init_var  816 v.1 0
>
> This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where  
> little endian has 2.
>
> I don't know what these numbers represent.
> I should look at other systems -- preferably a working 64 bit big  
> endian system, if there are any.
>
> Could be flags/type with text literals?
>
> (Btw, I always leave the TextLiteral.i3 change on my system,  
> lowering the maximum literal length to around 4gig even on 64 bit  
> targets, so 32 bit can bootstrap 64 bit...need to fix the compiler  
> here to do target arithmetic instead of host arithmetic...)
>
> Thanks,
>  - Jay
>
>
> From: hosking at cs.purdue.edu
> To: jay.krell at cornell.edu
> Date: Sat, 8 Nov 2008 22:55:54 +0000
> CC: m3devel at elegosoft.com
> Subject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
>
> Have you checked to make sure that the pre-gcc IR makes sense?  Does  
> it see these values as 32-bit or 64-bit?  The reason I ask is that  
> there may be something wrong in your m3middle target specs that  
> cause bad IR.  Once we know the IR looks right then we can try and  
> fix the problem.
>
> Antony Hosking | Associate Professor | Computer Science | Purdue  
> University
> 305 N. University Street | West Lafayette | IN 47907 | USA
> Office +1 765 494 6001 | Mobile +1 765 427 5484
>
>
>
> On 8 Nov 2008, at 15:45, Jay wrote:
>
>
> truncated as usual...along with losing a lot of whitespace..
> Let's see...
>
>
> Tony, how about switching these #if's?
> I explain some of why below.
>
> static void
> m3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t,  
> m3_type dst_T)
> {
>  tree val;
> #if 0
>  if (o != 0 || TREE_TYPE (v) != dst_t) {
>    v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),
>     bitsize_int (o));
>  }
> #else
>  /* failsafe, but inefficient */
>  if (o != 0 || TREE_TYPE (v) != dst_t) {
>    v = m3_build1 (ADDR_EXPR, t_addr, v);
>    v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));
>    v = m3_build1 (INDIRECT_REF, dst_t,
>     m3_cast (build_pointer_type (dst_t), v));
>  }
> #endif
>  TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing  
> problems */
>  val = m3_cast (src_t, EXPR_REF (-1));
>  if (src_T != dst_T) {
>    val = m3_build1 (CONVERT_EXPR, dst_t, val);
>  }
>  add_stmt (build2 (MODIFY_EXPR, dst_t, v, val));
>  EXPR_POP ();
> }
> m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t,  
> m3_type dst_T)
> {
> #if 0
>  if (o != 0 || TREE_TYPE (v) != src_t) {
>    v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),
>     bitsize_int (o));
>  }
> #else
>  /* failsafe, but inefficient */
>  if (o != 0 || TREE_TYPE (v) != src_t) {
>    v = m3_build1 (ADDR_EXPR, t_addr, v);
>    v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));
>    v = m3_build1 (INDIRECT_REF, src_t,
>     m3_cast (build_pointer_type (src_t), v));
>  }
> #endif
>  TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing  
> problems */
>  if (src_T != dst_T) {
>    v = m3_build1 (CONVERT_EXPR, dst_t, v);
>  }
>  EXPR_PUSH (v);
> }
>
> I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).
> In an early version it seems that many loads/stores of global  
> variables is wrong.
> The "first" code affected is RTLinker_ExpandModuleTable.
>
> PROCEDURE ExpandModuleTable () =
>  CONST InitialTableSize = 500;
>  VAR new_mods: ADDRESS;  n_bytes: INTEGER;
>  BEGIN
>    IF (modules = NIL) THEN
>      (* first time... *)
>      max_modules := InitialTableSize;
>      modules := Cstdlib.malloc (InitialTableSize * BYTESIZE  
> (RT0.ModulePtr));
>      IF (modules = NIL) THEN Cstdlib.abort (); END;
>    ELSE
>      n_bytes := max_modules * BYTESIZE (RT0.ModulePtr);
>      new_mods := Cstdlib.malloc (n_bytes + n_bytes);
>      IF (new_mods = NIL) THEN Cstdlib.abort (); END;
>      EVAL Cstring.memcpy (new_mods, modules, n_bytes);
>      Cstdlib.free (modules);
>      modules := new_mods;
>      INC (max_modules, max_modules);
>    END;
>  END ExpandModuleTable;
>
> It should read back modules as NIL.
> What happens is it reads back modules as not NIL,
> and then max_modules should also be zero, but is "large", leading to  
> malloc to fail,
> and abort called.
>
> The output of cm3cg -y is reasonable.
> It indicates offset loads.
>
> What is incorrect is that in many loads and stores, but not all, the
> offsets are lost.
>
> (923) begin_procedure
>  procedure RTLinker__ExpandModuleTable
> RTLinker__ExpandModuleTable(924) set_source_line
>  source line  208
> (925) load_nil
> (926) store
>  store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb
> (927) set_source_line
>  source line  207
> (928) set_source_line
>  source line  210
> (929) load_nil
> (930) load
>  m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xb
>
> Here is the incorrect code:
>
> .globl RTLinker__ExpandModuleTable
> .stabd 46,0,0
> .stabn 68,0,206,.LM108
> .LM108:
> .LFBB8:
> .set nomips16
> .ent RTLinker__ExpandModuleTable
> RTLinker__ExpandModuleTable:
> .frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0
> .mask 0xd0000000,-8
> .fmask 0x00000000,0
> daddiu $sp,$sp,-64
> sd $31,56($sp)
> sd $fp,48($sp)
> sd $28,40($sp)
> move $fp,$sp
> lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
> daddu $28,$28,$25
> daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
> .LBB9:
> .stabn 68,0,208,.LM109
> .LM109:
> sd $0,16($fp)
> .stabn 68,0,210,.LM110
> .LM110:
> dla $2,MM_RTLinker
> ld $2,0($2)      << WRONG
> bne $2,$0,.L61
> .stabn 68,0,212,.LM111
> .LM111:
> dla $3,MM_RTLinker
> ld $2,128($3)
> andi $2,$2,0x0
> ori $2,$2,0x1f4
> sd $2,128($3)
> .stabn 68,0,213,.LM112
> .LM112:
> li $4,4000   # 0xfa0
> jal malloc
> sd $2,0($fp)
> ld $4,0($fp)
> dla $3,MM_RTLinker
> ld $2,136($3)
> andi $2,$2,0x0
> or $2,$4,$2
> sd $2,136($3)     << OK, offset seems low, but it agrees with  
> elsewhere
> .stabn 68,0,214,.LM113
> .LM113:
> dla $2,MM_RTLinker
> ld $2,0($2)       << WRONG
> bne $2,$0,.L65
> jal abort
> b .L65
> .L61:
> .stabn 68,0,216,.LM114
> .LM114:
> dla $2,MM_RTLinker
> ld $2,0($2)        << WRONG
> dsll $2,$2,3
> sd $2,8($fp)
> .stabn 68,0,217,.LM115
> .LM115:
> ld $3,8($fp)
> ld $2,8($fp)
> daddu $2,$3,$2
> move $4,$2
> jal malloc
> sd $2,0($fp)
> ld $2,0($fp)
> sd $2,16($fp)
> .stabn 68,0,218,.LM116
> .LM116:
> ld $2,16($fp)
> bne $2,$0,.L64
> jal abort
> .L64:
> .stabn 68,0,219,.LM117
> .LM117:
> ld $4,16($fp)
> dla $2,MM_RTLinker
> ld $2,0($2)          << WRONG
> ld $3,8($fp)
> move $5,$2
> move $6,$3
> jal memcpy
> sd $2,0($fp)
> .stabn 68,0,220,.LM118
> .LM118:
> dla $2,MM_RTLinker
> ld $2,0($2)           << WRONG
> move $4,$2
> jal free
> .stabn 68,0,221,.LM119
> .LM119:
> ld $4,16($fp)
> dla $3,MM_RTLinker
> ld $2,136($3)         << OK
> andi $2,$2,0x0
> or $2,$4,$2
> sd $2,136($3)         << OK
> .stabn 68,0,222,.LM120
> .LM120:
> dla $2,MM_RTLinker
> ld $3,0($2)           << WRONG
> dla $2,MM_RTLinker
> ld $2,0($2)           << WRONG
> daddu $4,$3,$2
> dla $3,MM_RTLinker
> ld $2,128($3)         << OK
> andi $2,$2,0x0
> or $2,$2,$4
> sd $2,128($3)         << OK
> .L65:
> .LBE9:
> .stabn 68,0,224,.LM121
> .LM121:
> move $sp,$fp
> ld $31,56($sp)
> ld $fp,48($sp)
> ld $28,40($sp)
> daddiu $sp,$sp,64
> j $31
> .end RTLinker__ExpandModuleTable
>
> Here is corrected code:
>
> RTLinker__ExpandModuleTable:
> .frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0
> .mask 0xd0000000,-8
> .fmask 0x00000000,0
> daddiu $sp,$sp,-64
> sd $31,56($sp)
> sd $fp,48($sp)
> sd $28,40($sp)
> move $fp,$sp
> lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
> daddu $28,$28,$25
> daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
> .LBB9:
> .stabn 68,0,208,.LM109
> .LM109:
> sd $0,16($fp)
> .stabn 68,0,210,.LM110
> .LM110:
> dla $2,MM_RTLinker
> daddiu $2,$2,136     << CORRECT (offset seems low, but at least it  
> isn't zero)
> ld $2,0($2)
> bne $2,$0,.L61
> .stabn 68,0,212,.LM111
> .LM111:
> dla $2,MM_RTLinker
> daddiu $3,$2,128     << CORRECT
> li $2,500   # 0x1f4
> sd $2,0($3)
> .stabn 68,0,213,.LM112
> .LM112:
> li $4,4000   # 0xfa0
> jal malloc
> sd $2,0($fp)
> dla $2,MM_RTLinker
> daddiu $3,$2,136      << CORRECT
> ld $2,0($fp)
> sd $2,0($3)
> .stabn 68,0,214,.LM113
> .LM113:
> dla $2,MM_RTLinker
> daddiu $2,$2,136      << CORRECT
> ld $2,0($2)
> bne $2,$0,.L65
> jal abort
> b .L65
> .L61:
>
> I still dump core with this change, but I get much further.
> I have not tested this change at all otherwise -- i.e. on working  
> platforms.
>
> I have not debugged through the gcc/m3cg code to see why
> the offsets are only sometimes lost, or why other platforms
> have no problem. Is MIPS's "global pointer" unusual these days?
> I know SPARC has similar, and all the non-x86, non-AMD64 NT platforms
> had similar, but that's essentially nothing now (I'm not counting  
> IA64).
> I know Linux has PLT (procedure linkage table) and GOT (global  
> offset table).
> Same thing?
> I haven't looked much at other MIPS ports yet.
>
> http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8
>
> is what made it look about like it does -- in particularing using  
> BIT_FIELD_REF.
>
> 1.7:
>
> static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */
>     tree v;
>     int o;
>     tree src_t, dest_t;
>     m3_type src_T, dest_T;
> {
>  if (o == 0 && TREE_TYPE (v) == src_t) {
>    EXPR_PUSH (v);
>  } else {
>    tree adr = m3_build1 (ADDR_EXPR, t_addr, v);
>    if (o != 0) {
>      adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o /  
> BITS_PER_UNIT));
>    }
>    EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t,
>                          m3_cast (build_pointer_type (src_t), adr)));
>  }
> #if 1
>  if (src_T != dest_T) {
>    EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1));
>  }
>  if (debug_vars) {
>    const char *name = "noname";
>    if (v != 0 && DECL_NAME(v) != 0) {
>      name = IDENTIFIER_POINTER(DECL_NAME(v));
>    }
>    fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n",  
> name,
>            o, src_T, dest_T);
>  }
> #else
>  if (src_T != dest_T) {
>    if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) {
>      EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1));
>    } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) {
>      EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1));
>    } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) {
>      /* okay */
>    } else {
>      fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n",
>                  src_T, dest_T);
>    }
>  }
> #endif
> }
> static void
> m3_store (v, o, src_t, dest_t)  /* GCC32OK */
>     tree v;
>     int o;
>     tree src_t, dest_t;
> {
>  tree lhs, rhs;
>  if (TREE_TYPE (EXPR_REF (-1)) == src_t) {
>    rhs = EXPR_REF (-1);
>  } else {
>    rhs = m3_cast (src_t, EXPR_REF (-1));
>  }
>  if (o == 0 && TREE_TYPE (v) == dest_t) {
>    lhs = v;
>  } else {
>    tree f = make_node (FIELD_DECL);
>    TREE_TYPE (f) = dest_t;
>    DECL_ALIGN (f) = TYPE_ALIGN (dest_t);
>    DECL_SIZE (f) = TYPE_SIZE (dest_t);
>    DECL_MODE (f) = TYPE_MODE (dest_t);
>    DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT);
>    DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT);
>    DECL_FIELD_CONTEXT (f) = TREE_TYPE (v);
>    lhs = m3_build2 (COMPONENT_REF, dest_t, v, f);
>  }
> #if 1
>    expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0,  
> 0);
> #else
>  if (dest_t ==  NULL) {
>    expand_assignment (lhs, rhs, 0, 0);
>  } else {
>    expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0,  
> 0);
>  }
> #endif
>  EXPR_POP ();
> }
>
> 1.8:
> static void
> m3_load (tree v, int o,
>  tree src_t, m3_type src_T,
>  tree dst_t, m3_type dst_T) /* GCC32OK */
> {
>  if (o == 0 && TREE_TYPE (v) == src_t) {
>    EXPR_PUSH (v);
>  } else {
>    EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),
>     bitsize_int (o)));
>  }
>  if (src_T != dst_T) {
>    EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1));
>  }
>  if (option_vars_trace) {
>    const char *name = "noname";
>    if (v != 0 && DECL_NAME(v) != 0) {
>      name = IDENTIFIER_POINTER(DECL_NAME(v));
>    }
>    fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n",  
> name,
>     o, src_T, dst_T);
>  }
> }
> static void
> m3_store (tree v, int o, tree src_t, tree dst_t)  /* GCC32OK */
> {
>  tree lhs, rhs;
>  if (TREE_TYPE (EXPR_REF (-1)) == src_t) {
>    rhs = EXPR_REF (-1);
>  } else {
>    rhs = m3_cast (src_t, EXPR_REF (-1));
>  }
>  if (o == 0 && TREE_TYPE (v) == dst_t) {
>    lhs = v;
>  } else {
>    lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),
>       bitsize_int (o));
>  }
>  if (src_t != dst_t) {
>    rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs);
>  }
>  expand_assignment (lhs, rhs, 0, 0);
>  EXPR_POP ();
> }
>
> - Jay
>
> <AMD64_LINUX_RTLinker.mc><MIPS64_OPENBSD_RTLinker.mc>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081112/6950a8b4/attachment-0001.html>

From jay.krell at cornell.edu  Thu Nov 13 01:04:32 2008
From: jay.krell at cornell.edu (Jay)
Date: Thu, 13 Nov 2008 00:04:32 +0000
Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
In-Reply-To: <80AE04BD-4819-4085-8EBE-5996E8E6DB93@cs.purdue.edu>
References: <COL101-W3DB0BA622C4D31A82B602E6180@phx.gbl>
	<C47F6B5A-F154-4485-B839-96185B59E3B6@cs.purdue.edu>
	<COL101-W2CFF8666D1486BD514159E61B0@phx.gbl>
	<B9A95C1E-B873-40A9-A045-CD38E32CA5D0@cs.purdue.edu>
	<BAY116-W5161CD049D8CA0E7ABAE4AE6140@phx.gbl>
	<80AE04BD-4819-4085-8EBE-5996E8E6DB93@cs.purdue.edu>
Message-ID: <COL101-W60267BCFBF723F2491E3CEE6170@phx.gbl>


 > Jay, do you have access to an AMD64_DARWIN box?
 
No, sorry, I don't yet and not sure when I will.
Unless it is easy enough to run on non-Apple hardware.
I'll have to debug it (m3cg) then.
 
 - Jay



CC: m3devel at elegosoft.comFrom: hosking at cs.purdue.eduTo: jay.krell at cornell.eduSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Wed, 12 Nov 2008 20:46:53 +0000Jay, do you have access to an AMD64_DARWIN box?   I did get things working on that 64-bit platform.   Or is it an endianness issue you are dealing with.  In any case, I will not have a lot of bandwidth for doing much in the way of tackling this unless it becomes critical path for me soon.




Antony Hosking | Associate Professor | Computer Science | Purdue University
305 N. University Street | West Lafayette | IN 47907 | USA
Office +1 765 494 6001 | Mobile +1 765 427 5484


On 12 Nov 2008, at 07:57, Jay wrote:

Target.Little_endian is used to drive some target-specificity in the IR.   I guess, after all, that is why you have to specify it. The IR for RTLinker.m3 is identical for MIPS64_OPENBSD and SPARC64_SOLARIS, once I set them both to unaligned_procedures, which they both need. (Tangent, I ran m3cgcat on the .ms file a few times instead of the .mc file; that makes it crash.) I can't claim SPARC64_SOLARIS is any paragon of working and correct though, by far.I should also be able to compare with SPARC64_OPENBSD, I forgot about that, and didn't have an m3cg.I only ran SPARC64_SOLARIS far enough to discover it needed unaligned procedures.I think I should fork linux-common and solaris-common off of the minimal 29k openbsd-common, rather than worry about all the stuff in solaris-2-x and linux-libc, over 100k each. Then SPARC64_SOLARIS could/would be in better shape probably. MIPS64_OPENBSD cm3 gets further with unaligned_procedures, along with the parse.c fix -- it gets to a bus error in RunMainBody.Again an alignment problem, though I think maybe memory memory trashing.The fault occurs here:     desc_offset := init_depth * ADRSIZE (InitDesc);      desc := init_stack + desc_offset;      init_depth := desc.low_link; <<   InitDesc = RECORD    module   : RT0.ModulePtr;    low_link : INTEGER;  END;desc_offset appears to not be a multiple of 4. I will look into that. Long story short -- consider again the parse.c change?I admit I still haven't debugged it (or rather, the code that consumes the data produced by it).  - Jay

From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduCC: m3devel at elegosoft.comSubject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Sun, 9 Nov 2008 11:10:54 +0000I agree. I am suspicious too. I will investigate more. Thanks.  - Jay

From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 9 Nov 2008 10:01:58 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
Aren't the integer values supposed to be written endian-independently in the IR?  I am suspicious about these differences.




On 9 Nov 2008, at 01:25, Jay wrote:

Thanks for the tip/direction. Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the same" but for two systematic diffs:  - AMD64_LINUX has a bunch of added calls to check_nil.   This is a "bug" in it, inoptimal but ok code, because "first readable address" is 0. It should probably be 4k. I should write a program that iterates starting at 0 to find the value.  (or better yet, cm3 should have a -configure option that does this? -- for configuration values for which there is a safe default; in general, imho, the targets should be auto configured via C, assuming there is a native development environment available).  - some integers are different (again, systematically)  AMD64_LINUX: init_int  656 13 Int.64 init_chars  664 "    typecode " init_int  680 2 Int.64 init_var  688 v.1 0 init_int  696 13 Int.64 init_chars  704 "    typeid   " init_int  720 2 Int.64 init_var  728 v.1 0 init_int  736 13 Int.64 init_chars  744 "RunMainBody: " init_int  760 2 Int.64 init_var  768 v.1 0 init_int  776 19 Int.64 init_chars  784 "RunMainBody: exec: " init_int  808 2 Int.64 init_var  816 v.1 0 init_int  824 17 Int.64 init_chars  832 "ExpandInitStack: "MIPS64_OPENBSD:  init_chars  664 "    typecode " init_int  680 8796093022208 Int.64 init_var  688 v.1 0 init_int  696 13 Int.64 init_chars  704 "    typeid   " init_int  720 8796093022208 Int.64 init_var  728 v.1 0 init_int  736 13 Int.64 init_chars  744 "RunMainBody: " init_int  760 8796093022208 Int.64 init_var  768 v.1 0 init_int  776 19 Int.64 init_chars  784 "RunMainBody: exec: " init_int  808 8796093022208 Int.64 init_var  816 v.1 0This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where little endian has 2. I don't know what these numbers represent.I should look at other systems -- preferably a working 64 bit big endian system, if there are any. Could be flags/type with text literals? (Btw, I always leave the TextLiteral.i3 change on my system, lowering the maximum literal length to around 4gig even on 64 bit targets, so 32 bit can bootstrap 64 bit...need to fix the compiler here to do target arithmetic instead of host arithmetic...) Thanks, - Jay

From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sat, 8 Nov 2008 22:55:54 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Have you checked to make sure that the pre-gcc IR makes sense?  Does it see these values as 32-bit or 64-bit?  The reason I ask is that there may be something wrong in your m3middle target specs that cause bad IR.  Once we know the IR looks right then we can try and fix the problem.



Antony Hosking | Associate Professor | Computer Science | Purdue University
305 N. University Street | West Lafayette | IN 47907 | USA
Office +1 765 494 6001 | Mobile +1 765 427 5484


On 8 Nov 2008, at 15:45, Jay wrote:

truncated as usual...along with losing a lot of whitespace..Let's see...Tony, how about switching these #if's?I explain some of why below.static voidm3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){ tree val;#if 0 if (o != 0 || TREE_TYPE (v) != dst_t) {   v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),    bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) {   v = m3_build1 (ADDR_EXPR, t_addr, v);   v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));   v = m3_build1 (INDIRECT_REF, dst_t,    m3_cast (build_pointer_type (dst_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) {   val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP ();}m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){#if 0 if (o != 0 || TREE_TYPE (v) != src_t) {   v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),    bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) {   v = m3_build1 (ADDR_EXPR, t_addr, v);   v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));   v = m3_build1 (INDIRECT_REF, src_t,    m3_cast (build_pointer_type (src_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) {   v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v);}I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).In an early version it seems that many loads/stores of global variables is wrong.The "first" code affected is RTLinker_ExpandModuleTable.PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS;  n_bytes: INTEGER; BEGIN   IF (modules = NIL) THEN     (* first time... *)     max_modules := InitialTableSize;     modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr));     IF (modules = NIL) THEN Cstdlib.abort (); END;   ELSE     n_bytes := max_modules * BYTESIZE (RT0.ModulePtr);     new_mods := Cstdlib.malloc (n_bytes + n_bytes);     IF (new_mods = NIL) THEN Cstdlib.abort (); END;     EVAL Cstring.memcpy (new_mods, modules, n_bytes);     Cstdlib.free (modules);     modules := new_mods;     INC (max_modules, max_modules);   END; END ExpandModuleTable;It should read back modules as NIL.What happens is it reads back modules as not NIL,and then max_modules should also be zero, but is "large", leading to malloc to fail,and abort called.The output of cm3cg -y is reasonable.It indicates offset loads.What is incorrect is that in many loads and stores, but not all, theoffsets are lost.(923) begin_procedure procedure RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable(924) set_source_line source line  208(925) load_nil(926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb(927) set_source_line source line  207(928) set_source_line source line  210(929) load_nil(930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xbHere is the incorrect code:.globl RTLinker__ExpandModuleTable.stabd 46,0,0.stabn 68,0,206,.LM108.LM108:.LFBB8:.set nomips16.ent RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable:.frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerld $2,0($2)      << WRONGbne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $3,MM_RTLinkerld $2,128($3)andi $2,$2,0x0ori $2,$2,0x1f4sd $2,128($3).stabn 68,0,213,.LM112.LM112:li $4,4000   # 0xfa0jal mallocsd $2,0($fp)ld $4,0($fp)dla $3,MM_RTLinkerld $2,136($3)andi $2,$2,0x0or $2,$4,$2sd $2,136($3)     << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerld $2,0($2)       << WRONG bne $2,$0,.L65jal abortb .L65.L61:.stabn 68,0,216,.LM114.LM114:dla $2,MM_RTLinkerld $2,0($2)        << WRONG dsll $2,$2,3sd $2,8($fp).stabn 68,0,217,.LM115.LM115:ld $3,8($fp)ld $2,8($fp)daddu $2,$3,$2move $4,$2jal mallocsd $2,0($fp)ld $2,0($fp)sd $2,16($fp).stabn 68,0,218,.LM116.LM116:ld $2,16($fp)bne $2,$0,.L64jal abort.L64:.stabn 68,0,219,.LM117.LM117:ld $4,16($fp)dla $2,MM_RTLinkerld $2,0($2)          << WRONGld $3,8($fp)move $5,$2move $6,$3jal memcpysd $2,0($fp).stabn 68,0,220,.LM118.LM118:dla $2,MM_RTLinkerld $2,0($2)           << WRONGmove $4,$2jal free.stabn 68,0,221,.LM119.LM119:ld $4,16($fp)dla $3,MM_RTLinkerld $2,136($3)         << OKandi $2,$2,0x0or $2,$4,$2sd $2,136($3)         << OK .stabn 68,0,222,.LM120.LM120:dla $2,MM_RTLinkerld $3,0($2)           << WRONGdla $2,MM_RTLinkerld $2,0($2)           << WRONGdaddu $4,$3,$2dla $3,MM_RTLinkerld $2,128($3)         << OKandi $2,$2,0x0or $2,$2,$4sd $2,128($3)         << OK.L65:.LBE9:.stabn 68,0,224,.LM121.LM121:move $sp,$fpld $31,56($sp)ld $fp,48($sp)ld $28,40($sp)daddiu $sp,$sp,64j $31.end RTLinker__ExpandModuleTableHere is corrected code:RTLinker__ExpandModuleTable:.frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerdaddiu $2,$2,136     << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2)bne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $2,MM_RTLinkerdaddiu $3,$2,128     << CORRECT li $2,500   # 0x1f4sd $2,0($3).stabn 68,0,213,.LM112.LM112:li $4,4000   # 0xfa0jal mallocsd $2,0($fp)dla $2,MM_RTLinkerdaddiu $3,$2,136      << CORRECT ld $2,0($fp)sd $2,0($3).stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerdaddiu $2,$2,136      << CORRECT ld $2,0($2)bne $2,$0,.L65jal abortb .L65.L61:I still dump core with this change, but I get much further.I have not tested this change at all otherwise -- i.e. on working platforms.I have not debugged through the gcc/m3cg code to see whythe offsets are only sometimes lost, or why other platformshave no problem. Is MIPS's "global pointer" unusual these days?I know SPARC has similar, and all the non-x86, non-AMD64 NT platformshad similar, but that's essentially nothing now (I'm not counting IA64).I know Linux has PLT (procedure linkage table) and GOT (global offset table).Same thing?I haven't looked much at other MIPS ports yet.http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8is what made it look about like it does -- in particularing using BIT_FIELD_REF.1.7:static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */    tree v;    int o;    tree src_t, dest_t;    m3_type src_T, dest_T;{ if (o == 0 && TREE_TYPE (v) == src_t) {   EXPR_PUSH (v); } else {   tree adr = m3_build1 (ADDR_EXPR, t_addr, v);   if (o != 0) {     adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT));   }   EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t,                         m3_cast (build_pointer_type (src_t), adr))); }#if 1 if (src_T != dest_T) {   EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) {   const char *name = "noname";   if (v != 0 && DECL_NAME(v) != 0) {     name = IDENTIFIER_POINTER(DECL_NAME(v));   }   fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n", name,           o, src_T, dest_T); }#else if (src_T != dest_T) {   if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) {     EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1));   } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) {     EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1));   } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) {     /* okay */   } else {     fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n",                 src_T, dest_T);   } }#endif}static voidm3_store (v, o, src_t, dest_t)  /* GCC32OK */    tree v;    int o;    tree src_t, dest_t;{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) {   rhs = EXPR_REF (-1); } else {   rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) {   lhs = v; } else {   tree f = make_node (FIELD_DECL);   TREE_TYPE (f) = dest_t;   DECL_ALIGN (f) = TYPE_ALIGN (dest_t);   DECL_SIZE (f) = TYPE_SIZE (dest_t);   DECL_MODE (f) = TYPE_MODE (dest_t);   DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT);   DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT);   DECL_FIELD_CONTEXT (f) = TREE_TYPE (v);   lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); }#if 1   expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);#else if (dest_t ==  NULL) {   expand_assignment (lhs, rhs, 0, 0); } else {   expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); }#endif EXPR_POP ();}1.8:static voidm3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */{ if (o == 0 && TREE_TYPE (v) == src_t) {   EXPR_PUSH (v); } else {   EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),    bitsize_int (o))); } if (src_T != dst_T) {   EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) {   const char *name = "noname";   if (v != 0 && DECL_NAME(v) != 0) {     name = IDENTIFIER_POINTER(DECL_NAME(v));   }   fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n", name,    o, src_T, dst_T); }}static voidm3_store (tree v, int o, tree src_t, tree dst_t)  /* GCC32OK */{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) {   rhs = EXPR_REF (-1); } else {   rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) {   lhs = v; } else {   lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),      bitsize_int (o)); } if (src_t != dst_t) {   rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP ();}- Jay<AMD64_LINUX_RTLinker.mc><MIPS64_OPENBSD_RTLinker.mc>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081113/2f8b22da/attachment-0001.html>

From rodney.bates at wichita.edu  Thu Nov 13 16:45:34 2008
From: rodney.bates at wichita.edu (Rodney M. Bates)
Date: Thu, 13 Nov 2008 09:45:34 -0600
Subject: [M3devel] Still can't bootstrap cm3
In-Reply-To: <20081112081855.alr1p7k4ggw0k4k0@mail.elegosoft.com>
References: <4919ED6D.6000700@wichita.edu>
	<20081112081855.alr1p7k4ggw0k4k0@mail.elegosoft.com>
Message-ID: <491C4B9E.4010307@wichita.edu>

Olaf Wagner wrote:
> 
> it's not easy to understand what has gone wrong at this point.
> Usually, if you have a working system, cm3/scripts/upgrade.sh
> should be able to bootstrap to a new set of sources. This is tested
> every day in the tinderbox release tests, too. As the last builds seem
> to have succeeded, I don't think there is a real problem in our
> code base currently. You may want to have a look at test_build_current
> in cm3/scripts/regression/defs.sh to see what's done in the regression
> builds.
> 
> I'd suggest you either start again from the last release or
> one of the latest snapshots available from the build machine.

Yes, that's what I always end up doing.  But it would be nice to be
able someday to get my local compiler to rebuild itself :-(.

But even that is not working for me.  Reinstalling a snapshot from
2008-6-6 and a fresh cvs checkout, upgrade.sh gives the missing
.M3EXPORTS failure, and do-cm3-core.sh (after realclean) gets bad
version stamps on Compiler.Platform and Compiler.ThisPlatform,
while compiling SocketPosix.m3.

Going to the most recent (at the time) snapshot of 2008-11-12,
do-cm3-core.sh will run without failures, but leaves a compiler
that always gets the undefined ROOT failure, even trying to compile
a trivial program.

> I assume you are not building a cross compiler, but just trying
> to perform an upgrade.

That's right.  The only reason I messed with the cross scripts was that
I hoped they might take care of the necessary bootstrapping in a way
the would work.

> 
> Daily snapshots are available from
> http://www.opencm3.net/snaps/snapshot-index.html
> 
> I hope this helps,
> 
> Olaf
> 
>


From jay.krell at cornell.edu  Thu Nov 13 17:37:47 2008
From: jay.krell at cornell.edu (Jay)
Date: Thu, 13 Nov 2008 16:37:47 +0000
Subject: [M3devel] Still can't bootstrap cm3
In-Reply-To: <491C4B9E.4010307@wichita.edu>
References: <4919ED6D.6000700@wichita.edu>
	<20081112081855.alr1p7k4ggw0k4k0@mail.elegosoft.com> 
	<491C4B9E.4010307@wichita.edu>
Message-ID: <BAY118-W1318AE445EC7D9ACF911E8E6170@phx.gbl>


Randy, ROOT not being defined is a function of your config files.
Look at mine. Mine are willing to ferry it from an environment variable.
The various scripts figure it out and set it on the command line.
 e.g. sysinfo.sh
 
Granted..I don't think ROOT is supposed to be required.
ROOT is the root of the source tree, as you correctly?successfully worked around.
You are supposed to be able to use the compiler without a source tree.
But you can't, um, build the source tree without a source tree.
 
I am very tempted to have cm3.exe set ROOT itself,
but it is a bit inappropriate actually.
 
PERHAPS if it was renamed CM3_ROOT...
 
Olaf isn't saying "just use a current release", he is
saying "start with a current release" and then try to build locally.
 
You cannot necessarily bootstrap from arbitrarily old releases.
But you should definitely be able to bootstrap from current snapshots.
 
It does work.
 
Whenever you get into a "bad state" -- after any failed run,
I encourage you to blow away the entire cm3 install and "start over".
 
Several "partial builds" will leave you with a "mixed state".
 
Some of the scripts do or used to "backup" the pkg store,
but I don't think it was worth the effort...the effort I put
in to clone the .sh in .cmd. I think I punted doing that in the *.py,
and I don't miss it.
 
I also recommend trying the *.py files.
 
> that always gets the undefined ROOT failure, even trying to compile> a trivial program.
Hm. Could be your config file wants it?
Could be my config files wants it?
I guess the usual refrain -- send us your config file(s).
I can dig more later..maybe days..busy with other stuff..
 
 
 - Jay> Date: Thu, 13 Nov 2008 09:45:34 -0600> From: rodney> To: m3devel> Subject: Re: [M3devel] Still can't bootstrap cm3> > Olaf wrote:> > > > it's not easy to understand what has gone wrong at this point.> > Usually, if you have a working system, cm3/scripts/upgrade.sh> > should be able to bootstrap to a new set of sources. This is tested> > every day in the tinderbox release tests, too. As the last builds seem> > to have succeeded, I don't think there is a real problem in our> > code base currently. You may want to have a look at test_build_current> > in cm3/scripts/regression/defs.sh to see what's done in the regression> > builds.> > > > I'd suggest you either start again from the last release or> > one of the latest snapshots available from the build machine.> > Yes, that's what I always end up doing. But it would be nice to be> able someday to get my local compiler to rebuild itself :-(.> > But even that is not working for me. Reinstalling a snapshot from> 2008-6-6 and a fresh cvs checkout, upgrade.sh gives the missing> .M3EXPORTS failure, and do-cm3-core.sh (after realclean) gets bad> version stamps on Compiler.Platform and Compiler.ThisPlatform,> while compiling SocketPosix.m3.> > Going to the most recent (at the time) snapshot of 2008-11-12,> do-cm3-core.sh will run without failures, but leaves a compiler> that always gets the undefined ROOT failure, even trying to compile> a trivial program.> > > I assume you are not building a cross compiler, but just trying> > to perform an upgrade.> > That's right. The only reason I messed with the cross scripts was that> I hoped they might take care of the necessary bootstrapping in a way> the would work.> > > > > Daily snapshots are available from> > http://www.opencm3.net/snaps/snapshot-index.html> > > > I hope this helps,> > > > Olaf> > > >
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081113/3998c75f/attachment-0001.html>

From jay.krell at cornell.edu  Thu Nov 13 17:54:55 2008
From: jay.krell at cornell.edu (Jay)
Date: Thu, 13 Nov 2008 16:54:55 +0000
Subject: [M3devel] Still can't bootstrap cm3
In-Reply-To: <BAY118-W1318AE445EC7D9ACF911E8E6170@phx.gbl>
References: <4919ED6D.6000700@wichita.edu>
	<20081112081855.alr1p7k4ggw0k4k0@mail.elegosoft.com> 
	<491C4B9E.4010307@wichita.edu> 
	<BAY118-W1318AE445EC7D9ACF911E8E6170@phx.gbl>
Message-ID: <BAY117-W27AE620521647E466EC8D2E6170@phx.gbl>


Also, until you find something that works:
 export OMIT_GCC=yes
 
Otherwise you are going to waste a lot more time experimenting.
 
 - Jay



From: jay.krell at cornell.eduTo: rodney.bates at wichita.edu; m3devel at elegosoft.comDate: Thu, 13 Nov 2008 16:37:47 +0000Subject: Re: [M3devel] Still can't bootstrap cm3

Randy, ROOT not being defined is a function of your config files.Look at mine. Mine are willing to ferry it from an environment variable.The various scripts figure it out and set it on the command line. e.g. sysinfo.sh Granted..I don't think ROOT is supposed to be required.ROOT is the root of the source tree, as you correctly?successfully worked around.You are supposed to be able to use the compiler without a source tree.But you can't, um, build the source tree without a source tree. I am very tempted to have cm3.exe set ROOT itself,but it is a bit inappropriate actually. PERHAPS if it was renamed CM3_ROOT... Olaf isn't saying "just use a current release", he issaying "start with a current release" and then try to build locally. You cannot necessarily bootstrap from arbitrarily old releases.But you should definitely be able to bootstrap from current snapshots. It does work. Whenever you get into a "bad state" -- after any failed run,I encourage you to blow away the entire cm3 install and "start over". Several "partial builds" will leave you with a "mixed state". Some of the scripts do or used to "backup" the pkg store,but I don't think it was worth the effort...the effort I putin to clone the .sh in .cmd. I think I punted doing that in the *.py,and I don't miss it. I also recommend trying the *.py files. > that always gets the undefined ROOT failure, even trying to compile> a trivial program.Hm. Could be your config file wants it?Could be my config files wants it?I guess the usual refrain -- send us your config file(s).I can dig more later..maybe days..busy with other stuff..   - Jay> Date: Thu, 13 Nov 2008 09:45:34 -0600> From: rodney> To: m3devel> Subject: Re: [M3devel] Still can't bootstrap cm3> > Olaf wrote:> > > > it's not easy to understand what has gone wrong at this point.> > Usually, if you have a working system, cm3/scripts/upgrade.sh> > should be able to bootstrap to a new set of sources. This is tested> > every day in the tinderbox release tests, too. As the last builds seem> > to have succeeded, I don't think there is a real problem in our> > code base currently. You may want to have a look at test_build_current> > in cm3/scripts/regression/defs.sh to see what's done in the regression> > builds.> > > > I'd suggest you either start again from the last release or> > one of the latest snapshots available from the build machine.> > Yes, that's what I always end up doing. But it would be nice to be> able someday to get my local compiler to rebuild itself :-(.> > But even that is not working for me. Reinstalling a snapshot from> 2008-6-6 and a fresh cvs checkout, upgrade.sh gives the missing> .M3EXPORTS failure, and do-cm3-core.sh (after realclean) gets bad> version stamps on Compiler.Platform and Compiler.ThisPlatform,> while compiling SocketPosix.m3.> > Going to the most recent (at the time) snapshot of 2008-11-12,> do-cm3-core.sh will run without failures, but leaves a compiler> that always gets the undefined ROOT failure, even trying to compile> a trivial program.> > > I assume you are not building a cross compiler, but just trying> > to perform an upgrade.> > That's right. The only reason I messed with the cross scripts was that> I hoped they might take care of the necessary bootstrapping in a way> the would work.> > > > > Daily snapshots are available from> > http://www.opencm3.net/snaps/snapshot-index.html> > > > I hope this helps,> > > > Olaf> > > >
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081113/00c9851a/attachment-0001.html>

From hosking at cs.purdue.edu  Thu Nov 13 18:06:22 2008
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Thu, 13 Nov 2008 11:06:22 -0600
Subject: [M3devel] Still can't bootstrap cm3
In-Reply-To: <491C4B9E.4010307@wichita.edu>
References: <4919ED6D.6000700@wichita.edu>
	<20081112081855.alr1p7k4ggw0k4k0@mail.elegosoft.com>
	<491C4B9E.4010307@wichita.edu>
Message-ID: <77D02692-31FE-4326-8E0E-8758C323CDE9@cs.purdue.edu>

This is odd.  I've been rebuilding my local compiler by hand several  
times in the last few weeks.  I will try the scripts and follow up.

Antony Hosking | Associate Professor | Computer Science | Purdue  
University
305 N. University Street | West Lafayette | IN 47907 | USA
Office +1 765 494 6001 | Mobile +1 765 427 5484



On 13 Nov 2008, at 09:45, Rodney M. Bates wrote:

> Olaf Wagner wrote:
>> it's not easy to understand what has gone wrong at this point.
>> Usually, if you have a working system, cm3/scripts/upgrade.sh
>> should be able to bootstrap to a new set of sources. This is tested
>> every day in the tinderbox release tests, too. As the last builds  
>> seem
>> to have succeeded, I don't think there is a real problem in our
>> code base currently. You may want to have a look at  
>> test_build_current
>> in cm3/scripts/regression/defs.sh to see what's done in the  
>> regression
>> builds.
>> I'd suggest you either start again from the last release or
>> one of the latest snapshots available from the build machine.
>
> Yes, that's what I always end up doing.  But it would be nice to be
> able someday to get my local compiler to rebuild itself :-(.
>
> But even that is not working for me.  Reinstalling a snapshot from
> 2008-6-6 and a fresh cvs checkout, upgrade.sh gives the missing
> .M3EXPORTS failure, and do-cm3-core.sh (after realclean) gets bad
> version stamps on Compiler.Platform and Compiler.ThisPlatform,
> while compiling SocketPosix.m3.
>
> Going to the most recent (at the time) snapshot of 2008-11-12,
> do-cm3-core.sh will run without failures, but leaves a compiler
> that always gets the undefined ROOT failure, even trying to compile
> a trivial program.
>
>> I assume you are not building a cross compiler, but just trying
>> to perform an upgrade.
>
> That's right.  The only reason I messed with the cross scripts was  
> that
> I hoped they might take care of the necessary bootstrapping in a way
> the would work.
>
>> Daily snapshots are available from
>> http://www.opencm3.net/snaps/snapshot-index.html
>> I hope this helps,
>> Olaf
>>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081113/d9f9e670/attachment-0001.html>

From rodney.bates at wichita.edu  Fri Nov 14 00:29:17 2008
From: rodney.bates at wichita.edu (Rodney M. Bates)
Date: Thu, 13 Nov 2008 17:29:17 -0600
Subject: [M3devel] The undefined ROOT problem, rebuilding cm3
Message-ID: <491CB84D.4000008@wichita.edu>

Here is how the undefined ROOT problem happens when trying to rebuild
cm3 with itself.

I had been making the assumption that commands build, ship, clean, when
supplied to the do-cm3-*.sh scripts, did the same things as -build,
-ship, -clean supplied to the cm3 command.  Not so.

build actually does cm3 -build -override, as do buildlocal and empty
commands to the scripts. clean does cm3 -clean -override, as does cleanlocal.
There is a buildglobal command that does -build without -override, but
it also does a ship as well.  There appears to be no way to just build,
without either overriding or shipping.

So, I have been running do-cm3-*.sh with no argument (and sometimes build).
This starts a chain of events that eventually leads to an installed libm3
that always executes quake script bundle-ov.tmpl, and it requires that
either M3BUNDLE or ROOT be defined, neither of which normally is.  This
will happen trying to compile any program that uses libm3, which is just
about everything.

We should document this somewhere.


From jay.krell at cornell.edu  Fri Nov 14 02:44:44 2008
From: jay.krell at cornell.edu (Jay)
Date: Fri, 14 Nov 2008 01:44:44 +0000
Subject: [M3devel] The undefined ROOT problem, rebuilding cm3
In-Reply-To: <491CB84D.4000008@wichita.edu>
References: <491CB84D.4000008@wichita.edu>
Message-ID: <COL101-W79AA2376232A9B60AC3D33E6160@phx.gbl>


I don't know, but something doesn't seem right.
 
D:\cm3>dir /s/b bund* *ov* *temp* *tmp*...
D:\cm3\pkg\libm3\src\bundleintf\bundle.tmpl
 
if defined("M3BUNDLE")  _M3BUNDLE = M3BUNDLEelse  _M3BUNDLE = BIN_USE & SL & "m3bundle"end
D:\dev2\cm3.2>dir /s/b /a-d bund*tmp*D:\dev2\cm3.2\doc\help\gen_html\libm3\src\bundleintf\bundle.tmpl.htmlD:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle-ov.tmplD:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle.tmpl
 
:\dev2\cm3.2>dir /s/b /a-d bund*tmp*:\dev2\cm3.2\doc\help\gen_html\libm3\src\bundleintf\bundle.tmpl.html:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle-ov.tmpl:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle.tmpl
 
bundle-ov.tmp:
 
if defined("M3BUNDLE")  _M3BUNDLE = M3BUNDLEelse  _M3BUNDLE = format(ROOT & SL & "m3-tools" & SL & "m3bundle" & SL & "%s" &                     SL & "m3bundle", BUILD_DIR)  M3BUNDLE = _M3BUNDLEend
 => this file requires ROOT or M3BUNDLE like you say, but this isn't normally used.
 
D:\dev2\cm3.2>dir /s/b m3makefile | findstr /i bundleintfD:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\m3makefile
 
if defined("_M3BUNDLE_OVERRIDE")  template  ("bundle-ov")endtemplate  ("bundle")
You defined _M3BUNDLE_OVERRIDE somewhere along the way?
 
Try replacing the one file with the other..
 
 - Jay> Date: Thu, 13 Nov 2008 17:29:17 -0600> From: rodney.bates at wichita.edu> To: m3devel at elegosoft.com> Subject: [M3devel] The undefined ROOT problem, rebuilding cm3> > Here is how the undefined ROOT problem happens when trying to rebuild> cm3 with itself.> > I had been making the assumption that commands build, ship, clean, when> supplied to the do-cm3-*.sh scripts, did the same things as -build,> -ship, -clean supplied to the cm3 command. Not so.> > build actually does cm3 -build -override, as do buildlocal and empty> commands to the scripts. clean does cm3 -clean -override, as does cleanlocal.> There is a buildglobal command that does -build without -override, but> it also does a ship as well. There appears to be no way to just build,> without either overriding or shipping.> > So, I have been running do-cm3-*.sh with no argument (and sometimes build).> This starts a chain of events that eventually leads to an installed libm3> that always executes quake script bundle-ov.tmpl, and it requires that> either M3BUNDLE or ROOT be defined, neither of which normally is. This> will happen trying to compile any program that uses libm3, which is just> about everything.> > We should document this somewhere.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081114/3c45491f/attachment-0001.html>

From rodney.bates at wichita.edu  Fri Nov 14 03:01:30 2008
From: rodney.bates at wichita.edu (Rodney M. Bates)
Date: Thu, 13 Nov 2008 20:01:30 -0600
Subject: [M3devel] The undefined ROOT problem, rebuilding cm3
In-Reply-To: <COL101-W79AA2376232A9B60AC3D33E6160@phx.gbl>
References: <491CB84D.4000008@wichita.edu>
	<COL101-W79AA2376232A9B60AC3D33E6160@phx.gbl>
Message-ID: <491CDBFA.7030500@wichita.edu>

Jay wrote:
> I don't know, but something doesn't seem right.
>  
> D:\cm3>dir /s/b bund* *ov* *temp* *tmp*
> ...
> D:\cm3\pkg\libm3\src\bundleintf\bundle.tmpl
>  
> if defined("M3BUNDLE")
>   _M3BUNDLE = M3BUNDLE
> else
>   _M3BUNDLE = BIN_USE & SL & "m3bundle"
> end
> 
> D:\dev2\cm3.2>dir /s/b /a-d bund*tmp*
> D:\dev2\cm3.2\doc\help\gen_html\libm3\src\bundleintf\bundle.tmpl.html
> D:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle-ov.tmpl
> D:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle.tmpl
>  
> :\dev2\cm3.2>dir /s/b /a-d bund*tmp*
> :\dev2\cm3.2\doc\help\gen_html\libm3\src\bundleintf\bundle.tmpl.html
> :\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle-ov.tmpl
> :\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle.tmpl
>  
> bundle-ov.tmp:
>  
> if defined("M3BUNDLE")
>   _M3BUNDLE = M3BUNDLE
> else
>   _M3BUNDLE = format(ROOT & SL & "m3-tools" & SL & "m3bundle" & SL & "%s" &
>                      SL & "m3bundle", BUILD_DIR)
>   M3BUNDLE = _M3BUNDLE
> end
> 
>  => this file requires ROOT or M3BUNDLE like you say, but this isn't 
> normally used.
>  
> D:\dev2\cm3.2>dir /s/b m3makefile | findstr /i bundleintf
> D:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\m3makefile
>  
> if defined("_M3BUNDLE_OVERRIDE")
>   template  ("bundle-ov")
> end
> template  ("bundle")
> 
> You defined _M3BUNDLE_OVERRIDE somewhere along the way?

Maybe I should have given more detail on the chain of events.
You got most of it.  The missing piece is:

m3-libs/libm3/src/m3overrides contains:
_M3BUNDLE_OVERRIDE = "T"

So, 'do-cm3-*.sh build' runs 'cm3 -build -override', causing the
above file of quake code to be interpreted, defining _M3BUNDLE_OVERRIDE,
and the rest is as you have noted.

>  
> Try replacing the one file with the other..
>  
>  - Jay
> 
> 
> 
>  > Date: Thu, 13 Nov 2008 17:29:17 -0600
>  > From: rodney.bates at wichita.edu
>  > To: m3devel at elegosoft.com
>  > Subject: [M3devel] The undefined ROOT problem, rebuilding cm3
>  >
>  > Here is how the undefined ROOT problem happens when trying to rebuild
>  > cm3 with itself.
>  >
>  > I had been making the assumption that commands build, ship, clean, when
>  > supplied to the do-cm3-*.sh scripts, did the same things as -build,
>  > -ship, -clean supplied to the cm3 command. Not so.
>  >
>  > build actually does cm3 -build -override, as do buildlocal and empty
>  > commands to the scripts. clean does cm3 -clean -override, as does 
> cleanlocal.
>  > There is a buildglobal command that does -build without -override, but
>  > it also does a ship as well. There appears to be no way to just build,
>  > without either overriding or shipping.
>  >
>  > So, I have been running do-cm3-*.sh with no argument (and sometimes 
> build).
>  > This starts a chain of events that eventually leads to an installed libm3
>  > that always executes quake script bundle-ov.tmpl, and it requires that
>  > either M3BUNDLE or ROOT be defined, neither of which normally is. This
>  > will happen trying to compile any program that uses libm3, which is just
>  > about everything.
>  >
>  > We should document this somewhere.
> 



From hosking at cs.purdue.edu  Fri Nov 14 03:50:07 2008
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Thu, 13 Nov 2008 20:50:07 -0600
Subject: [M3devel] The undefined ROOT problem, rebuilding cm3
In-Reply-To: <491CDBFA.7030500@wichita.edu>
References: <491CB84D.4000008@wichita.edu>
	<COL101-W79AA2376232A9B60AC3D33E6160@phx.gbl>
	<491CDBFA.7030500@wichita.edu>
Message-ID: <74C195D7-3B89-4696-AB57-DE41DDD09E6D@cs.purdue.edu>

I generally run with buildship for these commands.

On 13 Nov 2008, at 20:01, Rodney M. Bates wrote:

> Jay wrote:
>> I don't know, but something doesn't seem right.
>> D:\cm3>dir /s/b bund* *ov* *temp* *tmp*
>> ...
>> D:\cm3\pkg\libm3\src\bundleintf\bundle.tmpl
>> if defined("M3BUNDLE")
>>  _M3BUNDLE = M3BUNDLE
>> else
>>  _M3BUNDLE = BIN_USE & SL & "m3bundle"
>> end
>> D:\dev2\cm3.2>dir /s/b /a-d bund*tmp*
>> D:\dev2\cm3.2\doc\help\gen_html\libm3\src\bundleintf\bundle.tmpl.html
>> D:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle-ov.tmpl
>> D:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle.tmpl
>> :\dev2\cm3.2>dir /s/b /a-d bund*tmp*
>> :\dev2\cm3.2\doc\help\gen_html\libm3\src\bundleintf\bundle.tmpl.html
>> :\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle-ov.tmpl
>> :\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle.tmpl
>> bundle-ov.tmp:
>> if defined("M3BUNDLE")
>>  _M3BUNDLE = M3BUNDLE
>> else
>>  _M3BUNDLE = format(ROOT & SL & "m3-tools" & SL & "m3bundle" & SL &  
>> "%s" &
>>                     SL & "m3bundle", BUILD_DIR)
>>  M3BUNDLE = _M3BUNDLE
>> end
>> => this file requires ROOT or M3BUNDLE like you say, but this isn't  
>> normally used.
>> D:\dev2\cm3.2>dir /s/b m3makefile | findstr /i bundleintf
>> D:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\m3makefile
>> if defined("_M3BUNDLE_OVERRIDE")
>>  template  ("bundle-ov")
>> end
>> template  ("bundle")
>> You defined _M3BUNDLE_OVERRIDE somewhere along the way?
>
> Maybe I should have given more detail on the chain of events.
> You got most of it.  The missing piece is:
>
> m3-libs/libm3/src/m3overrides contains:
> _M3BUNDLE_OVERRIDE = "T"
>
> So, 'do-cm3-*.sh build' runs 'cm3 -build -override', causing the
> above file of quake code to be interpreted, defining  
> _M3BUNDLE_OVERRIDE,
> and the rest is as you have noted.
>
>> Try replacing the one file with the other..
>>  - Jay
>> > Date: Thu, 13 Nov 2008 17:29:17 -0600
>> > From: rodney.bates at wichita.edu
>> > To: m3devel at elegosoft.com
>> > Subject: [M3devel] The undefined ROOT problem, rebuilding cm3
>> >
>> > Here is how the undefined ROOT problem happens when trying to  
>> rebuild
>> > cm3 with itself.
>> >
>> > I had been making the assumption that commands build, ship,  
>> clean, when
>> > supplied to the do-cm3-*.sh scripts, did the same things as -build,
>> > -ship, -clean supplied to the cm3 command. Not so.
>> >
>> > build actually does cm3 -build -override, as do buildlocal and  
>> empty
>> > commands to the scripts. clean does cm3 -clean -override, as does  
>> cleanlocal.
>> > There is a buildglobal command that does -build without - 
>> override, but
>> > it also does a ship as well. There appears to be no way to just  
>> build,
>> > without either overriding or shipping.
>> >
>> > So, I have been running do-cm3-*.sh with no argument (and  
>> sometimes build).
>> > This starts a chain of events that eventually leads to an  
>> installed libm3
>> > that always executes quake script bundle-ov.tmpl, and it requires  
>> that
>> > either M3BUNDLE or ROOT be defined, neither of which normally is.  
>> This
>> > will happen trying to compile any program that uses libm3, which  
>> is just
>> > about everything.
>> >
>> > We should document this somewhere.



From wagner at elegosoft.com  Fri Nov 14 09:35:07 2008
From: wagner at elegosoft.com (Olaf Wagner)
Date: Fri, 14 Nov 2008 09:35:07 +0100
Subject: [M3devel] The undefined ROOT problem, rebuilding cm3
In-Reply-To: <491CB84D.4000008@wichita.edu>
References: <491CB84D.4000008@wichita.edu>
Message-ID: <20081114093507.rp91pc17kg8cc8gw@mail.elegosoft.com>

Quoting "Rodney M. Bates" <rodney.bates at wichita.edu>:

> Here is how the undefined ROOT problem happens when trying to rebuild
> cm3 with itself.
>
> I had been making the assumption that commands build, ship, clean, when
> supplied to the do-cm3-*.sh scripts, did the same things as -build,
> -ship, -clean supplied to the cm3 command.  Not so.
>
> build actually does cm3 -build -override, as do buildlocal and empty
> commands to the scripts. clean does cm3 -clean -override, as does cleanlocal.
> There is a buildglobal command that does -build without -override, but
> it also does a ship as well.  There appears to be no way to just build,
> without either overriding or shipping.
>
> So, I have been running do-cm3-*.sh with no argument (and sometimes build).
> This starts a chain of events that eventually leads to an installed libm3
> that always executes quake script bundle-ov.tmpl, and it requires that
> either M3BUNDLE or ROOT be defined, neither of which normally is.  This
> will happen trying to compile any program that uses libm3, which is just
> about everything.
>
> We should document this somewhere.

Perhaps you could write up some paragraphs for the scripts' README
and the web pages?
I always seem to overlook or tend to obscure some crucial points for
other users :-/

Olaf
-- 
Olaf Wagner -- elego Software Solutions GmbH
                Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany
phone: +49 30 23 45 86 96  mobile: +49 177 2345 869  fax: +49 30 23 45 86 95
    http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin
Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194



From jay.krell at cornell.edu  Fri Nov 14 12:08:17 2008
From: jay.krell at cornell.edu (Jay)
Date: Fri, 14 Nov 2008 11:08:17 +0000
Subject: [M3devel] The undefined ROOT problem, rebuilding cm3
In-Reply-To: <74C195D7-3B89-4696-AB57-DE41DDD09E6D@cs.purdue.edu>
References: <491CB84D.4000008@wichita.edu>
	<COL101-W79AA2376232A9B60AC3D33E6160@phx.gbl>
	<491CDBFA.7030500@wichita.edu>
	<74C195D7-3B89-4696-AB57-DE41DDD09E6D@cs.purdue.edu>
Message-ID: <COL101-W32D8D3B50ACB900B343094E6160@phx.gbl>


> From: hosking>
> I generally run with buildship for these commands.
 
That, plus if you "buildlocal", you are supposed to be barred from "shipping".
That is, if you build with -override, you aren't supposed to be able to "ship".
"ship" is to copy from the source tree to the install tree.
I think I did see a problem once where it let me ship "local" stuff.
 
 - Jay
 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081114/c160ff55/attachment-0001.html>

From rodney.bates at wichita.edu  Fri Nov 14 16:26:05 2008
From: rodney.bates at wichita.edu (Rodney M. Bates)
Date: Fri, 14 Nov 2008 09:26:05 -0600
Subject: [M3devel] The undefined ROOT problem, rebuilding cm3
In-Reply-To: <74C195D7-3B89-4696-AB57-DE41DDD09E6D@cs.purdue.edu>
References: <491CB84D.4000008@wichita.edu>
	<COL101-W79AA2376232A9B60AC3D33E6160@phx.gbl>
	<491CDBFA.7030500@wichita.edu>
	<74C195D7-3B89-4696-AB57-DE41DDD09E6D@cs.purdue.edu>
Message-ID: <491D988D.9080804@wichita.edu>

Tony Hosking wrote:
> I generally run with buildship for these commands.

And so will I, henceforth.

Rodney Bates




From jay.krell at cornell.edu  Fri Nov 14 21:24:55 2008
From: jay.krell at cornell.edu (Jay)
Date: Fri, 14 Nov 2008 20:24:55 +0000
Subject: [M3devel] Linux/amd64
Message-ID: <COL101-W20B494FEE8697CAC58E0DEE6160@phx.gbl>


I turned of gc just while running stubgen.
 
Leading to:
  http://m3.elegosoft.com/cm3/uploaded-archives/cm3-std-POSIX-AMD64_LINUX-d5.7.0.tar.gz  
  http://m3.elegosoft.com/cm3/uploaded-archives/cm3-min-POSIX-AMD64_LINUX-d5.7.0.tar.gz  

Please try them? See how well they work for you all?
I haven't run most of the stuff, but cm3 does build itself.
gc does usually?often work.

(saying "posix" I feel is redundant...)

 - Jay

From jay.krell at cornell.edu  Sun Nov 16 16:31:25 2008
From: jay.krell at cornell.edu (Jay)
Date: Sun, 16 Nov 2008 15:31:25 +0000
Subject: [M3devel] duplicated code?
Message-ID: <COL101-W82BC319326E633D918F12DE6100@phx.gbl>


Anyone want to clean up this duplicity?
 
 
D:\dev2\cm3.2>dir /s/b asttotype.m3  StubCode.m3D:\dev2\cm3.2\m3-comm\sharedobjgen\src\AstToType.m3D:\dev2\cm3.2\m3-comm\stubgen\src\AstToType.m3D:\dev2\cm3.2\m3-db\stablegen\src\AstToType.m3
D:\dev2\cm3.2\m3-comm\sharedobjgen\src\StubCode.m3D:\dev2\cm3.2\m3-comm\stubgen\src\StubCode.m3
 
somewhere in there, AMD64_LINUX tries to allocate a lot of memory, and fails
either there or soon thereafter.
The garbage collector is working.
 
 - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081116/deecc4a6/attachment-0001.html>

From jay.krell at cornell.edu  Sun Nov 16 17:18:40 2008
From: jay.krell at cornell.edu (Jay)
Date: Sun, 16 Nov 2008 16:18:40 +0000
Subject: [M3devel] duplicated code?
In-Reply-To: <COL101-W82BC319326E633D918F12DE6100@phx.gbl>
References: <COL101-W82BC319326E633D918F12DE6100@phx.gbl>
Message-ID: <COL101-W16C880C91AE709A4E8463FE6100@phx.gbl>


Er, the large allocation does come from the gc itself.
 
Tony, is there is an assumption that the heap is contiguous?That calls to RTOS.GetMemory return adjacent addresses?
 
This code allocates a large array:
 
    IF desc = NIL OR newSideSpan # NUMBER(desc^) THEN      WITH newDesc = NEW(UNTRACED REF ARRAY OF Desc, newSideSpan) DO
I'll know more shortly.
 
I guess..it looks like the heap can be discontiguous, but
we do record keeping for what it all spans.
 
The comments say:
(* The array desc and the global variables p0, and p1 describe the pages   that are part of the traced heap.  Either p0 and p1 are equal to Nil and   no pages are allocated; or both are valid pages and page p is allocated   iff
|          p0 <= p < p1|      AND desc[p - p0] != Unallocated
 
Hm..
 
Grow (0x44000) => 0x2b1e45256000   total: 1.5M
GetUntracedOpenArray(0x1a80)   span: 6.6M   density: 24%stubgen: Processing RemoteView.TGetUntracedOpenArray(0x3f0)t1:0xct2:0xat3:0x1Grow (0x52000) => 0x2aaaaaaab000   total: 1.8M
GetUntracedOpenArray(0x1ce69fc8)
 
I have GetUntracedOpenArray printing how many bytes it is asked for.
t1,t2,t3 are just the lengths of the strings being concatented.
Grow(x)=>y means Grow allocated x bytes at address y.
 
So now, these two addresses 0x2aaaaaaab000 and 0x2b1e45256000  are very far apart, like 400gig.
And it seems the heap allocator wants to allocate an array to describe the pages.
 
Hm. Page size is no longer tied to the underlying system -- no longer vm-tied gc.
Perhaps perhaps blowing it up, to say, 1meg, will address this?
 
But really, an array to describe pages spanning the results of separate memory allocations, seems wrong.
A sparser data structure would be good, that could describe arbitrary sized runs of pages as being in the same state.
 
 - Jay



From: jay.krell at cornell.eduTo: m3devel at elegosoft.comDate: Sun, 16 Nov 2008 15:31:25 +0000Subject: [M3devel] duplicated code?

Anyone want to clean up this duplicity?  D:\dev2\cm3.2>dir /s/b asttotype.m3  StubCode.m3D:\dev2\cm3.2\m3-comm\sharedobjgen\src\AstToType.m3D:\dev2\cm3.2\m3-comm\stubgen\src\AstToType.m3D:\dev2\cm3.2\m3-db\stablegen\src\AstToType.m3D:\dev2\cm3.2\m3-comm\sharedobjgen\src\StubCode.m3D:\dev2\cm3.2\m3-comm\stubgen\src\StubCode.m3 somewhere in there, AMD64_LINUX tries to allocate a lot of memory, and failseither there or soon thereafter.The garbage collector is working.  - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081116/a28dfe37/attachment-0001.html>

From hosking at cs.purdue.edu  Mon Nov 17 03:14:55 2008
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Sun, 16 Nov 2008 20:14:55 -0600
Subject: [M3devel] duplicated code?
In-Reply-To: <COL101-W16C880C91AE709A4E8463FE6100@phx.gbl>
References: <COL101-W82BC319326E633D918F12DE6100@phx.gbl>
	<COL101-W16C880C91AE709A4E8463FE6100@phx.gbl>
Message-ID: <4B6BDC53-DB07-4498-A0D4-CB383DFFD487@cs.purdue.edu>

On 16 Nov 2008, at 10:18, Jay wrote:

> Er, the large allocation does come from the gc itself.

What's being allocated?  The side array should not be too huge.

> Tony, is there is an assumption that the heap is contiguous?
> That calls to RTOS.GetMemory return adjacent addresses?

No assumption.

> This code allocates a large array:
>
>     IF desc = NIL OR newSideSpan # NUMBER(desc^) THEN
>       WITH newDesc = NEW(UNTRACED REF ARRAY OF Desc, newSideSpan) DO
>
> I'll know more shortly.
>
> I guess..it looks like the heap can be discontiguous, but
> we do record keeping for what it all spans.

Correct.

> The comments say:
> (* The array desc and the global variables p0, and p1 describe the  
> pages
>    that are part of the traced heap.  Either p0 and p1 are equal to  
> Nil and
>    no pages are allocated; or both are valid pages and page p is  
> allocated
>    iff
> |          p0 <= p < p1
> |      AND desc[p - p0] != Unallocated
>
>
> Hm..
>
> Grow (0x44000) => 0x2b1e45256000   total: 1.5M
> GetUntracedOpenArray(0x1a80)
>    span: 6.6M   density: 24%
> stubgen: Processing RemoteView.T
> GetUntracedOpenArray(0x3f0)
> t1:0xc
> t2:0xa
> t3:0x1
> Grow (0x52000) => 0x2aaaaaaab000   total: 1.8M
> GetUntracedOpenArray(0x1ce69fc8)
>
> I have GetUntracedOpenArray printing how many bytes it is asked for.
> t1,t2,t3 are just the lengths of the strings being concatented.
> Grow(x)=>y means Grow allocated x bytes at address y.
>
> So now, these two addresses 0x2aaaaaaab000 and 0x2b1e45256000  are  
> very far apart, like 400gig.
> And it seems the heap allocator wants to allocate an array to  
> describe the pages.

Ah, yes, that is most unfortunate.


> Hm. Page size is no longer tied to the underlying system -- no  
> longer vm-tied gc.
> Perhaps perhaps blowing it up, to say, 1meg, will address this?

I am working on minimizing the need for the global array, but we do  
need something that can be easily indexed like this.  Perhaps we need  
to pass a hint to RTOS.GetMemory that will try to allocate its regions  
close together.

> But really, an array to describe pages spanning the results of  
> separate memory allocations, seems wrong.
> A sparser data structure would be good, that could describe  
> arbitrary sized runs of pages as being in the same state.

Indeed.  As mentioned above, I am working to eliminate the need for  
this.  Code that starts us on this path will be checked in within a  
day or so.

>  - Jay
>
>
>
> From: jay.krell at cornell.edu
> To: m3devel at elegosoft.com
> Date: Sun, 16 Nov 2008 15:31:25 +0000
> Subject: [M3devel] duplicated code?
>
> Anyone want to clean up this duplicity?
>
>
> D:\dev2\cm3.2>dir /s/b asttotype.m3  StubCode.m3
> D:\dev2\cm3.2\m3-comm\sharedobjgen\src\AstToType.m3
> D:\dev2\cm3.2\m3-comm\stubgen\src\AstToType.m3
> D:\dev2\cm3.2\m3-db\stablegen\src\AstToType.m3
> D:\dev2\cm3.2\m3-comm\sharedobjgen\src\StubCode.m3
> D:\dev2\cm3.2\m3-comm\stubgen\src\StubCode.m3
>
>
> somewhere in there, AMD64_LINUX tries to allocate a lot of memory,  
> and fails
> either there or soon thereafter.
> The garbage collector is working.
>
>  - Jay
>
>
>
>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081116/e6f957c5/attachment-0001.html>

From jay.krell at cornell.edu  Mon Nov 17 09:39:48 2008
From: jay.krell at cornell.edu (Jay)
Date: Mon, 17 Nov 2008 08:39:48 +0000
Subject: [M3devel] AMD64_LINUX (again)
Message-ID: <COL101-W10D65D8CA3093DB21EB5D1E6130@phx.gbl>


ok, with the sbrk workaround now:
 
 
http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-std-AMD64_LINUX-d5.7.0.tar.gz
 You don't have to build anything.
 
http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-min-AMD64_LINUX-d5.7.0.tar.gz
  You probably want to run ./do-cm3-std.py buildship.
 
http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-AMD64_LINUX.gz
  You get to build your own of almost everything (./install-config.py && ./do-pkg.py m3cc buildship && ./do-cm3-front.py buildship && ./install-front.py && ./do-cm3-std.py buildship)
 
 
OR you can bootstrap from an existing other system, roughly:
 cd m3-sys/m3cc && cm3 -DM3CC_TARGET=AMD64_LINUX && cd scripts/python && ./do-cm3-front.py AMD64_LINUX boot && ./boot1.py AMD64_LINUX and copy /cm3-boot-AMD64_LINUX-1.tar.gz to target machine and extract it, cd into it, make, and then you have native cm3 and proceed to previous..
 
 
I'm able to run Juno, Cube, Calculator, Mentor.
 
Hm. formsedit crashes. Darn.
 
Program received signal SIGSEGV, Segmentation fault.[Switching to Thread 1140881760 (LWP 13270)]0x00002afc180b9d6a in ScrollerVBTClass__PaintViewWithShadows (M3_Ao3CED_v=Cannot access memory at address 0x144006bf8)    at ../src/lego/POSIX/ScrollerVBTClass.m3:325325         VBT.PaintTint(v, r, v.shadow.bg);(gdb) cContinuing.Cannot find user-level thread for LWP 13242: generic error(gdb) bt#0  0x00002afc180b9d6a in ScrollerVBTClass__PaintViewWithShadows (M3_Ao3CED_v=Cannot access memory at address 0x144006bf8
) at ../src/lego/POSIX/ScrollerVBTClass.m3:325Cannot fetch general-purpose registers for thread 1140881760: generic error(gdb)
 
 
still might be worth other people trying out..
 
 
 - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081117/d6aecb25/attachment-0001.html>

From wagner at elegosoft.com  Mon Nov 17 10:10:43 2008
From: wagner at elegosoft.com (Olaf Wagner)
Date: Mon, 17 Nov 2008 10:10:43 +0100
Subject: [M3devel] The undefined ROOT problem, rebuilding cm3
In-Reply-To: <491CB84D.4000008@wichita.edu>
References: <491CB84D.4000008@wichita.edu>
Message-ID: <20081117101043.m08cyzrq8gkksco8@mail.elegosoft.com>

Quoting "Rodney M. Bates" <rodney.bates at wichita.edu>:

> Here is how the undefined ROOT problem happens when trying to rebuild
> cm3 with itself.
>
> I had been making the assumption that commands build, ship, clean, when
> supplied to the do-cm3-*.sh scripts, did the same things as -build,
> -ship, -clean supplied to the cm3 command.  Not so.
>
> build actually does cm3 -build -override, as do buildlocal and empty
> commands to the scripts. clean does cm3 -clean -override, as does cleanlocal.
> There is a buildglobal command that does -build without -override, but
> it also does a ship as well.  There appears to be no way to just build,
> without either overriding or shipping.
>
> So, I have been running do-cm3-*.sh with no argument (and sometimes build).
> This starts a chain of events that eventually leads to an installed libm3
> that always executes quake script bundle-ov.tmpl, and it requires that
> either M3BUNDLE or ROOT be defined, neither of which normally is.  This
> will happen trying to compile any program that uses libm3, which is just
> about everything.
>
> We should document this somewhere.

It seems that this time my email wasn't complete, for whatever reasons.
I was out of town for some days so I noticed only today.

Rodney, I did ask you if you could write up some paragraphs for the
README in scripts and the web pages that clarifies this matter so that
others won't run into it again?

I always seem to miss some crucial points when trying to explain the
build system myself...

Thanks in advance,

Olaf
-- 
Olaf Wagner -- elego Software Solutions GmbH
                Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany
phone: +49 30 23 45 86 96  mobile: +49 177 2345 869  fax: +49 30 23 45 86 95
    http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin
Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194



From wagner at elegosoft.com  Mon Nov 17 10:30:00 2008
From: wagner at elegosoft.com (Olaf Wagner)
Date: Mon, 17 Nov 2008 10:30:00 +0100
Subject: [M3devel] The undefined ROOT problem, rebuilding cm3
In-Reply-To: <491CB84D.4000008@wichita.edu>
References: <491CB84D.4000008@wichita.edu>
Message-ID: <20081117103000.ld2m2zsu8w48o4ck@mail.elegosoft.com>

Hm, now I seem to be the victim of mail truncation. I'll  try a top-quote
this time (our system administrator will investigate this). Third attempt:

It seems that this time my email wasn't complete, for whatever reasons.
I was out of town for some days so I noticed only today.

Rodney, I did ask you if you could write up some paragraphs for the
README in scripts and the web pages that clarifies this matter so that
others won't run into it again?

I always seem to miss some crucial points when trying to explain the
build system myself...

Thanks in advance,

Olaf

Quoting "Rodney M. Bates" <rodney.bates at wichita.edu>:

> Here is how the undefined ROOT problem happens when trying to rebuild
> cm3 with itself.
>
> I had been making the assumption that commands build, ship, clean, when
> supplied to the do-cm3-*.sh scripts, did the same things as -build,
> -ship, -clean supplied to the cm3 command.  Not so.
>
> build actually does cm3 -build -override, as do buildlocal and empty
> commands to the scripts. clean does cm3 -clean -override, as does cleanlocal.
> There is a buildglobal command that does -build without -override, but
> it also does a ship as well.  There appears to be no way to just build,
> without either overriding or shipping.
>
> So, I have been running do-cm3-*.sh with no argument (and sometimes build).
> This starts a chain of events that eventually leads to an installed libm3
> that always executes quake script bundle-ov.tmpl, and it requires that
> either M3BUNDLE or ROOT be defined, neither of which normally is.  This
> will happen trying to compile any program that uses libm3, which is just
> about everything.
>
> We should document this somewhere.



-- 
Olaf Wagner -- elego Software Solutions GmbH
                Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany
phone: +49 30 23 45 86 96  mobile: +49 177 2345 869  fax: +49 30 23 45 86 95
    http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin
Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194



From jay.krell at cornell.edu  Mon Nov 17 10:35:18 2008
From: jay.krell at cornell.edu (Jay)
Date: Mon, 17 Nov 2008 09:35:18 +0000
Subject: [M3devel] gc vs. large spans of memory
In-Reply-To: <4B6BDC53-DB07-4498-A0D4-CB383DFFD487@cs.purdue.edu>
References: <COL101-W82BC319326E633D918F12DE6100@phx.gbl>
	<COL101-W16C880C91AE709A4E8463FE6100@phx.gbl> 
	<4B6BDC53-DB07-4498-A0D4-CB383DFFD487@cs.purdue.edu>
Message-ID: <BAY120-W65F83A8674A6D4949CA7CE6130@phx.gbl>


1meg pages weren't large enough to avoid running out of memory.
 
 
Any thoughts on using the boehm-gc?
I don't know if it is compacting.
Or even generational.
Or anything about it really, just that it is much ported and presumably well maintained, since it is part of "gcc", part of the gcc Java support.
 
 
Hm... I assume the right data structure here is a multi level array, that you index by picking off progressively less significant bits of an address (or page number) just like the hardware page tables for virtual memory (but without the perf benefits of a cache and using physical addresses after the first level).
When an entire level is at the same state, due to skipping a large span, you just have one shared entry for all of them, major savings.
 
 
Either that, or a binary searched array where you only store entries for state changes.
You'd use like the STL's upper_bound/lower_bound/equal_range functions, that is, really the same thing as bsearch, but when an entry isn't found, you return where it would be inserted, which is like the last place you looked before giving up.
 
It might be worth confirming "that I'm not crazy" -- that mmap behaves this way for other people, or that skimming its code suggests it is not surprising. Could be related to how malloc works also.
 
 - Jay



From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 16 Nov 2008 20:14:55 -0600CC: m3devel at elegosoft.comSubject: Re: [M3devel] duplicated code?


On 16 Nov 2008, at 10:18, Jay wrote:


Er, the large allocation does come from the gc itself.

What's being allocated?  The side array should not be too huge.

Tony, is there is an assumption that the heap is contiguous?That calls to RTOS.GetMemory return adjacent addresses?

No assumption.

This code allocates a large array:     IF desc = NIL OR newSideSpan # NUMBER(desc^) THEN      WITH newDesc = NEW(UNTRACED REF ARRAY OF Desc, newSideSpan) DOI'll know more shortly. I guess..it looks like the heap can be discontiguous, butwe do record keeping for what it all spans.

Correct.

The comments say:(* The array desc and the global variables p0, and p1 describe the pages   that are part of the traced heap.  Either p0 and p1 are equal to Nil and   no pages are allocated; or both are valid pages and page p is allocated   iff|          p0 <= p < p1|      AND desc[p - p0] != Unallocated Hm.. Grow (0x44000) => 0x2b1e45256000   total: 1.5MGetUntracedOpenArray(0x1a80)   span: 6.6M   density: 24%stubgen: Processing RemoteView.TGetUntracedOpenArray(0x3f0)t1:0xct2:0xat3:0x1Grow (0x52000) => 0x2aaaaaaab000   total: 1.8MGetUntracedOpenArray(0x1ce69fc8) I have GetUntracedOpenArray printing how many bytes it is asked for.t1,t2,t3 are just the lengths of the strings being concatented.Grow(x)=>y means Grow allocated x bytes at address y. So now, these two addresses 0x2aaaaaaab000 and 0x2b1e45256000  are very far apart, like 400gig.And it seems the heap allocator wants to allocate an array to describe the pages.

Ah, yes, that is most unfortunate.


Hm. Page size is no longer tied to the underlying system -- no longer vm-tied gc.Perhaps perhaps blowing it up, to say, 1meg, will address this?

I am working on minimizing the need for the global array, but we do need something that can be easily indexed like this.  Perhaps we need to pass a hint to RTOS.GetMemory that will try to allocate its regions close together.

But really, an array to describe pages spanning the results of separate memory allocations, seems wrong.A sparser data structure would be good, that could describe arbitrary sized runs of pages as being in the same state.

Indeed.  As mentioned above, I am working to eliminate the need for this.  Code that starts us on this path will be checked in within a day or so.

 - Jay

From: jay.krell at cornell.eduTo: m3devel at elegosoft.comDate: Sun, 16 Nov 2008 15:31:25 +0000Subject: [M3devel] duplicated code?Anyone want to clean up this duplicity?  D:\dev2\cm3.2>dir /s/b asttotype.m3  StubCode.m3D:\dev2\cm3.2\m3-comm\sharedobjgen\src\AstToType.m3D:\dev2\cm3.2\m3-comm\stubgen\src\AstToType.m3D:\dev2\cm3.2\m3-db\stablegen\src\AstToType.m3D:\dev2\cm3.2\m3-comm\sharedobjgen\src\StubCode.m3D:\dev2\cm3.2\m3-comm\stubgen\src\StubCode.m3 somewhere in there, AMD64_LINUX tries to allocate a lot of memory, and failseither there or soon thereafter.The garbage collector is working.  - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081117/8e6723b0/attachment-0001.html>

From hosking at cs.purdue.edu  Mon Nov 17 13:21:26 2008
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Mon, 17 Nov 2008 06:21:26 -0600
Subject: [M3devel] gc vs. large spans of memory
In-Reply-To: <BAY120-W65F83A8674A6D4949CA7CE6130@phx.gbl>
References: <COL101-W82BC319326E633D918F12DE6100@phx.gbl>
	<COL101-W16C880C91AE709A4E8463FE6100@phx.gbl>
	<4B6BDC53-DB07-4498-A0D4-CB383DFFD487@cs.purdue.edu>
	<BAY120-W65F83A8674A6D4949CA7CE6130@phx.gbl>
Message-ID: <AE6DC073-2F37-437A-8712-0C5F09B5A97D@cs.purdue.edu>

On 17 Nov 2008, at 03:35, Jay wrote:

> 1meg pages weren't large enough to avoid running out of memory.

Is this still the sidespan array problem?

> Any thoughts on using the boehm-gc?

Hell no.

> I don't know if it is compacting.

Indeed, Boehm's collector does not do compaction.  Also, Boehm's is  
fully conservative ("ambiguous-roots + ambiguous heap"), though it is  
possible to configure it to use precise heap information (which CM3  
does provide).

> Or even generational.

It is generational, and has a parallel collection mode, but it is not  
concurrent like ours (we have a better chance of scaling on SMPs).   
Our collector will soon also be "on-the-fly", which means we won't  
need to have a stop-the-world phase where all the threads are stopped  
at the same time to initiate GC.  Instead, we will simply signal  
threads one at a time to prepare for GC.

> Or anything about it really, just that it is much ported and  
> presumably well maintained, since it is part of "gcc", part of the  
> gcc Java support.

Our collector is much ported and well-maintained too! ;-)  Seriously,  
I am quite sure we have no collector bugs currently (it has been  
running reliably for many years now).  The problems you are  
encountering are porting issues and not bugs in the collector.  Also,  
the current collector is written in Modula-3, and nicely integrated  
with the Modula-3 object model and run-time.   There is a *huge*  
amount to be said for eating your own dog-food by writing the  
collector in Modula-3.  There is *no* good reason to step outside  
Modula-3 to C as would be needed for Boehm GC.

> Hm... I assume the right data structure here is a multi level array,  
> that you index by picking off progressively less significant bits of  
> an address (or page number) just like the hardware page tables for  
> virtual memory (but without the perf benefits of a cache and using  
> physical addresses after the first level).
> When an entire level is at the same state, due to skipping a large  
> span, you just have one shared entry for all of them, major savings.

Right.  I have this partially implemented already.

> Either that, or a binary searched array where you only store entries  
> for state changes.
> You'd use like the STL's upper_bound/lower_bound/equal_range  
> functions, that is, really the same thing as bsearch, but when an  
> entry isn't found, you return where it would be inserted, which is  
> like the last place you looked before giving up.

I'm concerned about the lookup costs for this.

> It might be worth confirming "that I'm not crazy" -- that mmap  
> behaves this way for other people, or that skimming its code  
> suggests it is not surprising. Could be related to how malloc works  
> also.

Are there hints that can be passed to mmap on that platform to cause  
less scattered mappings?  It is odd that sbrk is less scattered, since  
it must also bottom out at mmap, but perhaps it is trying to maintain  
a reasonably compact allocation below the "brk".

>
>
>  - Jay
>
>
> From: hosking at cs.purdue.edu
> To: jay.krell at cornell.edu
> Date: Sun, 16 Nov 2008 20:14:55 -0600
> CC: m3devel at elegosoft.com
> Subject: Re: [M3devel] duplicated code?
>
>
> On 16 Nov 2008, at 10:18, Jay wrote:
>
> Er, the large allocation does come from the gc itself.
>
> What's being allocated?  The side array should not be too huge.
>
> Tony, is there is an assumption that the heap is contiguous?
> That calls to RTOS.GetMemory return adjacent addresses?
>
> No assumption.
>
> This code allocates a large array:
>
>     IF desc = NIL OR newSideSpan # NUMBER(desc^) THEN
>       WITH newDesc = NEW(UNTRACED REF ARRAY OF Desc, newSideSpan) DO
>
> I'll know more shortly.
>
> I guess..it looks like the heap can be discontiguous, but
> we do record keeping for what it all spans.
>
> Correct.
>
> The comments say:
> (* The array desc and the global variables p0, and p1 describe the  
> pages
>    that are part of the traced heap.  Either p0 and p1 are equal to  
> Nil and
>    no pages are allocated; or both are valid pages and page p is  
> allocated
>    iff
> |          p0 <= p < p1
> |      AND desc[p - p0] != Unallocated
>
>
> Hm..
>
> Grow (0x44000) => 0x2b1e45256000   total: 1.5M
> GetUntracedOpenArray(0x1a80)
>    span: 6.6M   density: 24%
> stubgen: Processing RemoteView.T
> GetUntracedOpenArray(0x3f0)
> t1:0xc
> t2:0xa
> t3:0x1
> Grow (0x52000) => 0x2aaaaaaab000   total: 1.8M
> GetUntracedOpenArray(0x1ce69fc8)
>
> I have GetUntracedOpenArray printing how many bytes it is asked for.
> t1,t2,t3 are just the lengths of the strings being concatented.
> Grow(x)=>y means Grow allocated x bytes at address y.
>
> So now, these two addresses 0x2aaaaaaab000 and 0x2b1e45256000  are  
> very far apart, like 400gig.
> And it seems the heap allocator wants to allocate an array to  
> describe the pages.
>
> Ah, yes, that is most unfortunate.
>
>
> Hm. Page size is no longer tied to the underlying system -- no  
> longer vm-tied gc.
> Perhaps perhaps blowing it up, to say, 1meg, will address this?
>
> I am working on minimizing the need for the global array, but we do  
> need something that can be easily indexed like this.  Perhaps we  
> need to pass a hint to RTOS.GetMemory that will try to allocate its  
> regions close together.
>
> But really, an array to describe pages spanning the results of  
> separate memory allocations, seems wrong.
> A sparser data structure would be good, that could describe  
> arbitrary sized runs of pages as being in the same state.
>
> Indeed.  As mentioned above, I am working to eliminate the need for  
> this.  Code that starts us on this path will be checked in within a  
> day or so.
>
>  - Jay
>
>
>
> From: jay.krell at cornell.edu
> To: m3devel at elegosoft.com
> Date: Sun, 16 Nov 2008 15:31:25 +0000
> Subject: [M3devel] duplicated code?
>
> Anyone want to clean up this duplicity?
>
>
> D:\dev2\cm3.2>dir /s/b asttotype.m3  StubCode.m3
> D:\dev2\cm3.2\m3-comm\sharedobjgen\src\AstToType.m3
> D:\dev2\cm3.2\m3-comm\stubgen\src\AstToType.m3
> D:\dev2\cm3.2\m3-db\stablegen\src\AstToType.m3
> D:\dev2\cm3.2\m3-comm\sharedobjgen\src\StubCode.m3
> D:\dev2\cm3.2\m3-comm\stubgen\src\StubCode.m3
>
>
> somewhere in there, AMD64_LINUX tries to allocate a lot of memory,  
> and fails
> either there or soon thereafter.
> The garbage collector is working.
>
>  - Jay
>
>
>
>
>
>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081117/70cfa516/attachment-0001.html>

From jay.krell at cornell.edu  Mon Nov 17 13:48:59 2008
From: jay.krell at cornell.edu (Jay)
Date: Mon, 17 Nov 2008 12:48:59 +0000
Subject: [M3devel] gc vs. large spans of memory
In-Reply-To: <AE6DC073-2F37-437A-8712-0C5F09B5A97D@cs.purdue.edu>
References: <COL101-W82BC319326E633D918F12DE6100@phx.gbl>
	<COL101-W16C880C91AE709A4E8463FE6100@phx.gbl>
	<4B6BDC53-DB07-4498-A0D4-CB383DFFD487@cs.purdue.edu>
	<BAY120-W65F83A8674A6D4949CA7CE6130@phx.gbl>
	<AE6DC073-2F37-437A-8712-0C5F09B5A97D@cs.purdue.edu>
Message-ID: <COL101-W638E7324891BF366626DA4E6130@phx.gbl>


understood, ok.
 
I don't know the algorithm for allocating pages.
 
There is a "hint" parameter to mmap, it isn't exactly clear how it is handled.
Probably giving the block we have plus its size is reasonable.
I can try that out at some point, though using sbrk seems ok.
 
 - Jay



CC: m3devel at elegosoft.comFrom: hosking at cs.purdue.eduTo: jay.krell at cornell.eduSubject: Re: gc vs. large spans of memoryDate: Mon, 17 Nov 2008 06:21:26 -0600


On 17 Nov 2008, at 03:35, Jay wrote:


1meg pages weren't large enough to avoid running out of memory.

Is this still the sidespan array problem?

Any thoughts on using the boehm-gc?

Hell no.

I don't know if it is compacting.

Indeed, Boehm's collector does not do compaction.  Also, Boehm's is fully conservative ("ambiguous-roots + ambiguous heap"), though it is possible to configure it to use precise heap information (which CM3 does provide).

Or even generational.

It is generational, and has a parallel collection mode, but it is not concurrent like ours (we have a better chance of scaling on SMPs).  Our collector will soon also be "on-the-fly", which means we won't need to have a stop-the-world phase where all the threads are stopped at the same time to initiate GC.  Instead, we will simply signal threads one at a time to prepare for GC.

Or anything about it really, just that it is much ported and presumably well maintained, since it is part of "gcc", part of the gcc Java support.

Our collector is much ported and well-maintained too! ;-)  Seriously, I am quite sure we have no collector bugs currently (it has been running reliably for many years now).  The problems you are encountering are porting issues and not bugs in the collector.  Also, the current collector is written in Modula-3, and nicely integrated with the Modula-3 object model and run-time.   There is a *huge* amount to be said for eating your own dog-food by writing the collector in Modula-3.  There is *no* good reason to step outside Modula-3 to C as would be needed for Boehm GC.

Hm... I assume the right data structure here is a multi level array, that you index by picking off progressively less significant bits of an address (or page number) just like the hardware page tables for virtual memory (but without the perf benefits of a cache and using physical addresses after the first level).When an entire level is at the same state, due to skipping a large span, you just have one shared entry for all of them, major savings.

Right.  I have this partially implemented already.

Either that, or a binary searched array where you only store entries for state changes.You'd use like the STL's upper_bound/lower_bound/equal_range functions, that is, really the same thing as bsearch, but when an entry isn't found, you return where it would be inserted, which is like the last place you looked before giving up.

I'm concerned about the lookup costs for this.

It might be worth confirming "that I'm not crazy" -- that mmap behaves this way for other people, or that skimming its code suggests it is not surprising. Could be related to how malloc works also.

Are there hints that can be passed to mmap on that platform to cause less scattered mappings?  It is odd that sbrk is less scattered, since it must also bottom out at mmap, but perhaps it is trying to maintain a reasonably compact allocation below the "brk".

  - Jay

From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 16 Nov 2008 20:14:55 -0600CC: m3devel at elegosoft.comSubject: Re: [M3devel] duplicated code?


On 16 Nov 2008, at 10:18, Jay wrote:


Er, the large allocation does come from the gc itself.

What's being allocated?  The side array should not be too huge.

Tony, is there is an assumption that the heap is contiguous?That calls to RTOS.GetMemory return adjacent addresses?

No assumption.

This code allocates a large array:     IF desc = NIL OR newSideSpan # NUMBER(desc^) THEN      WITH newDesc = NEW(UNTRACED REF ARRAY OF Desc, newSideSpan) DOI'll know more shortly. I guess..it looks like the heap can be discontiguous, butwe do record keeping for what it all spans.

Correct.

The comments say:(* The array desc and the global variables p0, and p1 describe the pages   that are part of the traced heap.  Either p0 and p1 are equal to Nil and   no pages are allocated; or both are valid pages and page p is allocated   iff|          p0 <= p < p1|      AND desc[p - p0] != Unallocated Hm.. Grow (0x44000) => 0x2b1e45256000   total: 1.5MGetUntracedOpenArray(0x1a80)   span: 6.6M   density: 24%stubgen: Processing RemoteView.TGetUntracedOpenArray(0x3f0)t1:0xct2:0xat3:0x1Grow (0x52000) => 0x2aaaaaaab000   total: 1.8MGetUntracedOpenArray(0x1ce69fc8) I have GetUntracedOpenArray printing how many bytes it is asked for.t1,t2,t3 are just the lengths of the strings being concatented.Grow(x)=>y means Grow allocated x bytes at address y. So now, these two addresses 0x2aaaaaaab000 and 0x2b1e45256000  are very far apart, like 400gig.And it seems the heap allocator wants to allocate an array to describe the pages.

Ah, yes, that is most unfortunate.


Hm. Page size is no longer tied to the underlying system -- no longer vm-tied gc.Perhaps perhaps blowing it up, to say, 1meg, will address this?

I am working on minimizing the need for the global array, but we do need something that can be easily indexed like this.  Perhaps we need to pass a hint to RTOS.GetMemory that will try to allocate its regions close together.

But really, an array to describe pages spanning the results of separate memory allocations, seems wrong.A sparser data structure would be good, that could describe arbitrary sized runs of pages as being in the same state.

Indeed.  As mentioned above, I am working to eliminate the need for this.  Code that starts us on this path will be checked in within a day or so.

 - Jay

From: jay.krell at cornell.eduTo: m3devel at elegosoft.comDate: Sun, 16 Nov 2008 15:31:25 +0000Subject: [M3devel] duplicated code?Anyone want to clean up this duplicity?  D:\dev2\cm3.2>dir /s/b asttotype.m3  StubCode.m3D:\dev2\cm3.2\m3-comm\sharedobjgen\src\AstToType.m3D:\dev2\cm3.2\m3-comm\stubgen\src\AstToType.m3D:\dev2\cm3.2\m3-db\stablegen\src\AstToType.m3D:\dev2\cm3.2\m3-comm\sharedobjgen\src\StubCode.m3D:\dev2\cm3.2\m3-comm\stubgen\src\StubCode.m3 somewhere in there, AMD64_LINUX tries to allocate a lot of memory, and failseither there or soon thereafter.The garbage collector is working.  - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081117/4253a60f/attachment-0001.html>

From jay.krell at cornell.edu  Mon Nov 17 14:00:01 2008
From: jay.krell at cornell.edu (Jay)
Date: Mon, 17 Nov 2008 13:00:01 +0000
Subject: [M3devel] gc vs. large spans of memory
In-Reply-To: <AE6DC073-2F37-437A-8712-0C5F09B5A97D@cs.purdue.edu>
References: <COL101-W82BC319326E633D918F12DE6100@phx.gbl>
	<COL101-W16C880C91AE709A4E8463FE6100@phx.gbl>
	<4B6BDC53-DB07-4498-A0D4-CB383DFFD487@cs.purdue.edu>
	<BAY120-W65F83A8674A6D4949CA7CE6130@phx.gbl>
	<AE6DC073-2F37-437A-8712-0C5F09B5A97D@cs.purdue.edu> 
Message-ID: <COL101-W25E6863F4A5BFE2141AA24E6130@phx.gbl>


 > 1 meg pages didn't suffice
 > Is this still the sidespan array problem?
 
I assume so.
I just tried it out, got the out of memory, and gave up on that approach.
I guess I could bump it up to 1 gig or more, just to prove the theory but
of course at some point I'm squeamish about wasting address space or actual memory.
1gig is tiny for address space but not for actual memory.
(assuming a full 64 bit address space, which actually is probably not available by far)
 
 - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081117/2160cca5/attachment-0001.html>

From hosking at cs.purdue.edu  Mon Nov 17 19:31:01 2008
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Mon, 17 Nov 2008 12:31:01 -0600
Subject: [M3devel] gc vs. large spans of memory
In-Reply-To: <COL101-W25E6863F4A5BFE2141AA24E6130@phx.gbl>
References: <COL101-W82BC319326E633D918F12DE6100@phx.gbl>
	<COL101-W16C880C91AE709A4E8463FE6100@phx.gbl>
	<4B6BDC53-DB07-4498-A0D4-CB383DFFD487@cs.purdue.edu>
	<BAY120-W65F83A8674A6D4949CA7CE6130@phx.gbl>
	<AE6DC073-2F37-437A-8712-0C5F09B5A97D@cs.purdue.edu>
	<COL101-W25E6863F4A5BFE2141AA24E6130@phx.gbl>
Message-ID: <D67B5251-57BB-48F2-90E1-7F626B808FEA@cs.purdue.edu>

If sbrk works let's leave it at that for now.

On 17 Nov 2008, at 07:00, Jay wrote:

>  > 1 meg pages didn't suffice
>  > Is this still the sidespan array problem?
>
> I assume so.
> I just tried it out, got the out of memory, and gave up on that  
> approach.
> I guess I could bump it up to 1 gig or more, just to prove the  
> theory but
> of course at some point I'm squeamish about wasting address space or  
> actual memory.
> 1gig is tiny for address space but not for actual memory.
> (assuming a full 64 bit address space, which actually is probably  
> not available by far)
>
>  - Jay

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081117/c441cd1d/attachment-0001.html>

From rodney.bates at wichita.edu  Sun Nov 23 19:06:18 2008
From: rodney.bates at wichita.edu (Rodney M. Bates)
Date: Sun, 23 Nov 2008 12:06:18 -0600
Subject: [M3devel] The undefined ROOT problem, rebuilding cm3
In-Reply-To: <20081117103000.ld2m2zsu8w48o4ck@mail.elegosoft.com>
References: <491CB84D.4000008@wichita.edu>
	<20081117103000.ld2m2zsu8w48o4ck@mail.elegosoft.com>
Message-ID: <49299B9A.7050103@wichita.edu>

Olaf Wagner wrote:
> Hm, now I seem to be the victim of mail truncation. I'll  try a top-quote
> this time (our system administrator will investigate this). Third attempt:
> 
> It seems that this time my email wasn't complete, for whatever reasons.
> I was out of town for some days so I noticed only today.
> 
> Rodney, I did ask you if you could write up some paragraphs for the
> README in scripts and the web pages that clarifies this matter so that
> others won't run into it again?

Done.

> 
> I always seem to miss some crucial points when trying to explain the
> build system myself...
> 
> Thanks in advance,
> 
> Olaf
> 
> Quoting "Rodney M. Bates" <rodney.bates at wichita.edu>:
> 
>> Here is how the undefined ROOT problem happens when trying to rebuild
>> cm3 with itself.
>>
>> I had been making the assumption that commands build, ship, clean, when
>> supplied to the do-cm3-*.sh scripts, did the same things as -build,
>> -ship, -clean supplied to the cm3 command.  Not so.
>>
>> build actually does cm3 -build -override, as do buildlocal and empty
>> commands to the scripts. clean does cm3 -clean -override, as does 
>> cleanlocal.
>> There is a buildglobal command that does -build without -override, but
>> it also does a ship as well.  There appears to be no way to just build,
>> without either overriding or shipping.
>>
>> So, I have been running do-cm3-*.sh with no argument (and sometimes build)



From michael.anderson at elego.de  Wed Nov 26 03:10:24 2008
From: michael.anderson at elego.de (Michael Anderson)
Date: Wed, 26 Nov 2008 03:10:24 +0100
Subject: [M3devel] compiling cvsup on Debian
Message-ID: <20081126031024.o8hbvsdkgo0kw884@mail.elegosoft.com>

Howdy there,

I'm trying to compile cvsup-snap-16.1h on Debian Etch with cm3, using  
this thread as
starting point:
https://mail.elegosoft.com/pipermail/m3devel/2007-June/000468.html

Thus far, I've:

- applied the patch
- changed line 744 of FileAttr.m3 to "tz: UNTRACED REF Utime.struct_timezone;"
- changed all IMPORT directives for "SchedulerPosix" to "Scheduler",  
as well as the
corresponding procedure calls as outlined here:

https://mail.elegosoft.com/pipermail/m3devel/2008-January/001210.html

Which that leaves me with just one teensy error:

"../src/SigHandler.m3", line 221: unknown qualification '.' (IOAlertWait)

which does go away if I change the IMPORT and procedure calls back to  
SchedulerPosix for
JUST SigHandler.m3.

Is it safe to use both "Scheduler" and "SchedulerPosix", or will  
Bad_Things happen?

Thanks,
mike


-- 
Michael Anderson
IT Services & Support

elego Software Solutions GmbH
Gustav-Meyer-Allee 25
Building 12.3 (BIG) room 227
13355 Berlin, Germany

phone +49 30 23 45 86 96      michael.anderson at elegosoft.com
fax   +49 30 23 45 86 95      http://www.elegosoft.com

Geschaeftsfuehrer: Olaf Wagner, Sitz Berlin
Amtsgericht Berlin-Charlottenburg, HRB 77719, USt-IdNr: DE163214194




From hosking at cs.purdue.edu  Wed Nov 26 17:03:52 2008
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Wed, 26 Nov 2008 10:03:52 -0600
Subject: [M3devel] compiling cvsup on Debian
In-Reply-To: <20081126031024.o8hbvsdkgo0kw884@mail.elegosoft.com>
References: <20081126031024.o8hbvsdkgo0kw884@mail.elegosoft.com>
Message-ID: <192844D9-5F3B-4E15-B2E9-C540D3EBD15B@cs.purdue.edu>

Should be just fine!

On 25 Nov 2008, at 20:10, Michael Anderson wrote:

> Howdy there,
>
> I'm trying to compile cvsup-snap-16.1h on Debian Etch with cm3,  
> using this thread as
> starting point:
> https://mail.elegosoft.com/pipermail/m3devel/2007-June/000468.html
>
> Thus far, I've:
>
> - applied the patch
> - changed line 744 of FileAttr.m3 to "tz: UNTRACED REF  
> Utime.struct_timezone;"
> - changed all IMPORT directives for "SchedulerPosix" to "Scheduler",  
> as well as the
> corresponding procedure calls as outlined here:
>
> https://mail.elegosoft.com/pipermail/m3devel/2008-January/001210.html
>
> Which that leaves me with just one teensy error:
>
> "../src/SigHandler.m3", line 221: unknown qualification  
> '.' (IOAlertWait)
>
> which does go away if I change the IMPORT and procedure calls back  
> to SchedulerPosix for
> JUST SigHandler.m3.
>
> Is it safe to use both "Scheduler" and "SchedulerPosix", or will  
> Bad_Things happen?
>
> Thanks,
> mike
>
>
> -- 
> Michael Anderson
> IT Services & Support
>
> elego Software Solutions GmbH
> Gustav-Meyer-Allee 25
> Building 12.3 (BIG) room 227
> 13355 Berlin, Germany
>
> phone +49 30 23 45 86 96      michael.anderson at elegosoft.com
> fax   +49 30 23 45 86 95      http://www.elegosoft.com
>
> Geschaeftsfuehrer: Olaf Wagner, Sitz Berlin
> Amtsgericht Berlin-Charlottenburg, HRB 77719, USt-IdNr: DE163214194
>
>



From rodney.bates at wichita.edu  Tue Nov  4 15:45:35 2008
From: rodney.bates at wichita.edu (Rodney M. Bates)
Date: Tue, 04 Nov 2008 08:45:35 -0600
Subject: [M3devel] New m3gdb documentation
Message-ID: <4910600F.7000009@wichita.edu>

I have new and greatly expanded documentation on m3gdb.  It is created
in docbook format, and I have managed to get it to convert to one-page
html.  I intend to do multi-page html and pdf also.

I am not sure where in the repository this should go, where to put html
files so they are available on the web, or what to do with mechanically
derived files in the repository.  I thought of creating a new subdirectory
doc/help/m3gdb, but I can only find how to get cvs to add a new ordinary
file, not a directory.

Also, I presume that Olaf will want to get the html to look like the other
pages, which some kind of style sheet should do, but I don't know where to
find that either.

Any help will be appreciated.

Rodney Bates


From wagner at elegosoft.com  Tue Nov  4 17:04:32 2008
From: wagner at elegosoft.com (Olaf Wagner)
Date: Tue, 04 Nov 2008 17:04:32 +0100
Subject: [M3devel] New m3gdb documentation
In-Reply-To: <4910600F.7000009@wichita.edu>
References: <4910600F.7000009@wichita.edu>
Message-ID: <20081104170432.1nn4yh39444sc48k@mail.elegosoft.com>

Quoting "Rodney M. Bates" <rodney.bates at wichita.edu>:

> I have new and greatly expanded documentation on m3gdb.  It is created
> in docbook format, and I have managed to get it to convert to one-page
> html.  I intend to do multi-page html and pdf also.

All documentation is checked-in below cm3/doc and cm3/www.
m3gdb should probably be integrated into doc.

> I am not sure where in the repository this should go, where to put html
> files so they are available on the web, or what to do with mechanically
> derived files in the repository.  I thought of creating a new subdirectory
> doc/help/m3gdb, but I can only find how to get cvs to add a new ordinary
> file, not a directory.

just use `cvs add <dir>'

> Also, I presume that Olaf will want to get the html to look like the other
> pages, which some kind of style sheet should do, but I don't know where to
> find that either.

The style sheet is normal.css at the top level.
You can reference it like this in the HTML header:

     <LINK HREF="normal.css" REL="stylesheet" TYPE="text/css">

It's checked in at cm3/www/normal.css.

When you've checked-in all your additions, we need to ship them
to the WWW server to make them visible. I can do that if you like,
just let me know when you're finished.

It's great that you're taking care of m3gdb and thus providing a
working debugger for all M3 users!

Regards,

Olaf

> Any help will be appreciated.
>
> Rodney Bates
-- 
Olaf Wagner -- elego Software Solutions GmbH
                Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany
phone: +49 30 23 45 86 96  mobile: +49 177 2345 869  fax: +49 30 23 45 86 95
    http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin
Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194



From rodney.bates at wichita.edu  Thu Nov  6 17:31:23 2008
From: rodney.bates at wichita.edu (Rodney M. Bates)
Date: Thu, 06 Nov 2008 10:31:23 -0600
Subject: [M3devel] New m3gdb documentation
Message-ID: <49131BDB.20903@wichita.edu>

There is a new and greatly expanded web page with documentation on m3gdb,
the Modula-3 debugger, located at:

   http://modula3.elegosoft.com/cm3/doc/help/m3gdb/m3gdb-onepage.html

It's also accessible with more indexing, through the normal Modula-3
web pages, for example, starting at:

   http://modula3.elegosoft.com/cm3

and clicking through:

   -> Documentation -> CM3 Operations Guide-> m3gdb

Rodney M. Bates




From jay.krell at cornell.edu  Sat Nov  8 16:40:50 2008
From: jay.krell at cornell.edu (Jay)
Date: Sat, 8 Nov 2008 15:40:50 +0000
Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
Message-ID: <COL101-W11F245FB5F2B72E83291B8E6180@phx.gbl>


Tony, how about switching these #if's?
I explain some of why below.

parse.c:

static void
m3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T)
{
  tree val;
#if 0
  if (o != 0 || TREE_TYPE (v) != dst_t) {
    v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),
     bitsize_int (o));
  }
#else
  /* failsafe, but inefficient */
  if (o != 0 || TREE_TYPE (v) != dst_t) {
    v = m3_build1 (ADDR_EXPR, t_addr, v);
    v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));
    v = m3_build1 (INDIRECT_REF, dst_t,
     m3_cast (build_pointer_type (dst_t), v));
  }
#endif
  TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */
  val = m3_cast (src_t, EXPR_REF (-1));
  if (src_T != dst_T) {
    val = m3_build1 (CONVERT_EXPR, dst_t, val);
  }
  add_stmt (build2 (MODIFY_EXPR, dst_t, v, val));
  EXPR_POP ();
}

m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T)
{
#if 0
  if (o != 0 || TREE_TYPE (v) != src_t) {
    v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),
     bitsize_int (o));
  }
#else
  /* failsafe, but inefficient */
  if (o != 0 || TREE_TYPE (v) != src_t) {
    v = m3_build1 (ADDR_EXPR, t_addr, v);
    v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));
    v = m3_build1 (INDIRECT_REF, src_t,
     m3_cast (build_pointer_type (src_t), v));
  }
#endif
  TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */
  if (src_T != dst_T) {
    v = m3_build1 (CONVERT_EXPR, dst_t, v);
  }
  EXPR_PUSH (v);
}

I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).

In an early version it seems that many loads/stores of global variables is wrong.

The first code affected is RTLinker_ExpandModuleTable.

PROCEDURE ExpandModuleTable () =
  CONST InitialTableSize = 500;
  VAR new_mods: ADDRESS;  n_bytes: INTEGER;
  BEGIN
    IF (modules = NIL) THEN
      (* first time... *)
      max_modules := InitialTableSize;
      modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr));
      IF (modules = NIL) THEN Cstdlib.abort (); END;
    ELSE
      n_bytes := max_modules * BYTESIZE (RT0.ModulePtr);
      new_mods := Cstdlib.malloc (n_bytes + n_bytes);
      IF (new_mods = NIL) THEN Cstdlib.abort (); END;
      EVAL Cstring.memcpy (new_mods, modules, n_bytes);
      Cstdlib.free (modules);
      modules := new_mods;
      INC (max_modules, max_modules);
    END;
  END ExpandModuleTable;

It should read back modules as NIL.
What happens is it reads back modules as not NIL,
and then max_modules should also be zero, but is "large", leading to malloc to fail,
and abort called.

The output of cm3cg -y is reasonable.
It indicates offset loads.

What is incorrect is that in many loads and stores, but not all, the
offsets are lost.

(923) begin_procedure
  procedure RTLinker__ExpandModuleTable
 RTLinker__ExpandModuleTable(924) set_source_line
  source line  208
(925) load_nil
(926) store
  store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb
(927) set_source_line
  source line  207
(928) set_source_line
  source line  210
(929) load_nil
(930) load
  m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xb

Here is the incorrect code:

 .globl RTLinker__ExpandModuleTable
 .stabd 46,0,0
 .stabn 68,0,206,.LM108
.LM108:
.LFBB8:
 .set nomips16
 .ent RTLinker__ExpandModuleTable
RTLinker__ExpandModuleTable:
 .frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0
 .mask 0xd0000000,-8
 .fmask 0x00000000,0
 daddiu $sp,$sp,-64
 sd $31,56($sp)
 sd $fp,48($sp)
 sd $28,40($sp)
 move $fp,$sp
 lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
 daddu $28,$28,$25
 daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
.LBB9:
 .stabn 68,0,208,.LM109
.LM109:
 sd $0,16($fp)
 .stabn 68,0,210,.LM110
.LM110:
 dla $2,MM_RTLinker
 ld $2,0($2)      << WRONG
 bne $2,$0,.L61
 .stabn 68,0,212,.LM111
.LM111:
 dla $3,MM_RTLinker
 ld $2,128($3)
 andi $2,$2,0x0
 ori $2,$2,0x1f4
 sd $2,128($3)
 .stabn 68,0,213,.LM112
.LM112:
 li $4,4000   # 0xfa0
 jal malloc
 sd $2,0($fp)
 ld $4,0($fp)
 dla $3,MM_RTLinker
 ld $2,136($3)
 andi $2,$2,0x0
 or $2,$4,$2
 sd $2,136($3)     << OK, offset seems low, but it agrees with elsewhere 
 .stabn 68,0,214,.LM113
.LM113:
 dla $2,MM_RTLinker
 ld $2,0($2)       << WRONG 
 bne $2,$0,.L65
 jal abort
 b .L65
.L61:
 .stabn 68,0,216,.LM114
.LM114:
 dla $2,MM_RTLinker
 ld $2,0($2)        << WRONG 
 dsll $2,$2,3
 sd $2,8($fp)
 .stabn 68,0,217,.LM115
.LM115:
 ld $3,8($fp)
 ld $2,8($fp)
 daddu $2,$3,$2
 move $4,$2
 jal malloc
 sd $2,0($fp)
 ld $2,0($fp)
 sd $2,16($fp)
 .stabn 68,0,218,.LM116
.LM116:
 ld $2,16($fp)
 bne $2,$0,.L64
 jal abort
.L64:
 .stabn 68,0,219,.LM117
.LM117:
 ld $4,16($fp)
 dla $2,MM_RTLinker
 ld $2,0($2)          << WRONG
 ld $3,8($fp)
 move $5,$2
 move $6,$3
 jal memcpy
 sd $2,0($fp)
 .stabn 68,0,220,.LM118
.LM118:
 dla $2,MM_RTLinker
 ld $2,0($2)           << WRONG
 move $4,$2
 jal free
 .stabn 68,0,221,.LM119
.LM119:
 ld $4,16($fp)
 dla $3,MM_RTLinker
 ld $2,136($3)         << OK
 andi $2,$2,0x0
 or $2,$4,$2
 sd $2,136($3)         << OK 
 .stabn 68,0,222,.LM120
.LM120:
 dla $2,MM_RTLinker
 ld $3,0($2)           << WRONG
 dla $2,MM_RTLinker
 ld $2,0($2)           << WRONG
 daddu $4,$3,$2
 dla $3,MM_RTLinker
 ld $2,128($3)         << OK
 andi $2,$2,0x0
 or $2,$2,$4
 sd $2,128($3)         << OK
.L65:
.LBE9:
 .stabn 68,0,224,.LM121
.LM121:
 move $sp,$fp
 ld $31,56($sp)
 ld $fp,48($sp)
 ld $28,40($sp)
 daddiu $sp,$sp,64
 j $31
 .end RTLinker__ExpandModuleTable

Here is corrected code:

RTLinker__ExpandModuleTable:
 .frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0
 .mask 0xd0000000,-8
 .fmask 0x00000000,0
 daddiu $sp,$sp,-64
 sd $31,56($sp)
 sd $fp,48($sp)
 sd $28,40($sp)
 move $fp,$sp
 lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
 daddu $28,$28,$25
 daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
.LBB9:
 .stabn 68,0,208,.LM109
.LM109:
 sd $0,16($fp)
 .stabn 68,0,210,.LM110
.LM110:
 dla $2,MM_RTLinker
 daddiu $2,$2,136     << CORRECT (offset seems low, but at least it isn't zero) 
 ld $2,0($2)
 bne $2,$0,.L61
 .stabn 68,0,212,.LM111
.LM111:
 dla $2,MM_RTLinker
 daddiu $3,$2,128     << CORRECT 
 li $2,500   # 0x1f4
 sd $2,0($3)
 .stabn 68,0,213,.LM112
.LM112:
 li $4,4000   # 0xfa0
 jal malloc
 sd $2,0($fp)
 dla $2,MM_RTLinker
 daddiu $3,$2,136      << CORRECT 
 ld $2,0($fp)
 sd $2,0($3)
 .stabn 68,0,214,.LM113
.LM113:
 dla $2,MM_RTLinker
 daddiu $2,$2,136      << CORRECT 
 ld $2,0($2)
 bne $2,$0,.L65
 jal abort
 b .L65
.L61:

I still dump core with this change, but I get much further.
I have not tested this change at all otherwise -- i.e. on working platforms.

I have not debugged through the gcc/m3cg code to see why
the offsets are only sometimes lost, or why other platforms
have no problem. Is MIPS's "global pointer" unusual these days?
I know SPARC has similar, and all the non-x86, non-AMD64 NT platforms
had similar, but that's essentially nothing now (I'm not counting IA64).
I know Linux has PLT (procedure linkage table) and GOT (global offset table).
Same thing?
I haven't looked much at other MIPS ports yet.

http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8

is what made it look about like it does -- in particularing using BIT_FIELD_REF.

1.7:

static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */
     tree v;
     int o;
     tree src_t, dest_t;
     m3_type src_T, dest_T;
{
  if (o == 0 && TREE_TYPE (v) == src_t) {
    EXPR_PUSH (v);
  } else {
    tree adr = m3_build1 (ADDR_EXPR, t_addr, v);
    if (o != 0) {
      adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT));
    }
    EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t,
                          m3_cast (build_pointer_type (src_t), adr)));
  }
#if 1
  if (src_T != dest_T) {
    EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1));
  }
  if (debug_vars) {
    const char *name = "noname";
    if (v != 0 && DECL_NAME(v) != 0) {
      name = IDENTIFIER_POINTER(DECL_NAME(v));
    }
    fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n", name,
            o, src_T, dest_T);
  }
#else
  if (src_T != dest_T) {
    if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) {
      EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1));
    } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) {
      EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1));
    } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) {
      /* okay */
    } else {
      fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n",
                  src_T, dest_T);
    }
  }
#endif
}
static void
m3_store (v, o, src_t, dest_t)  /* GCC32OK */
     tree v;
     int o;
     tree src_t, dest_t;
{
  tree lhs, rhs;
  if (TREE_TYPE (EXPR_REF (-1)) == src_t) {
    rhs = EXPR_REF (-1);
  } else {
    rhs = m3_cast (src_t, EXPR_REF (-1));
  }
  if (o == 0 && TREE_TYPE (v) == dest_t) {
    lhs = v;
  } else {
    tree f = make_node (FIELD_DECL);
    TREE_TYPE (f) = dest_t;
    DECL_ALIGN (f) = TYPE_ALIGN (dest_t);
    DECL_SIZE (f) = TYPE_SIZE (dest_t);
    DECL_MODE (f) = TYPE_MODE (dest_t);
    DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT);
    DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT);
    DECL_FIELD_CONTEXT (f) = TREE_TYPE (v);
    lhs = m3_build2 (COMPONENT_REF, dest_t, v, f);
  }
#if 1
    expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);
#else
  if (dest_t ==  NULL) {
    expand_assignment (lhs, rhs, 0, 0);
  } else {
    expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);
  }
#endif
  EXPR_POP ();
}

1.8:
static void
m3_load (tree v, int o,
  tree src_t, m3_type src_T,
  tree dst_t, m3_type dst_T) /* GCC32OK */
{
  if (o == 0 && TREE_TYPE (v) == src_t) {
    EXPR_PUSH (v);
  } else {
    EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),
     bitsize_int (o)));
  }
  if (src_T != dst_T) {
    EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1));
  }
  if (option_vars_trace) {
    const char *name = "noname";
    if (v != 0 && DECL_NAME(v) != 0) {
      name = IDENTIFIER_POINTER(DECL_NAME(v));
    }
    fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n", name,
     o, src_T, dst_T);
  }
}
static void
m3_store (tree v, int o, tree src_t, tree dst_t)  /* GCC32OK */
{
  tree lhs, rhs;
  if (TREE_TYPE (EXPR_REF (-1)) == src_t) {
    rhs = EXPR_REF (-1);
  } else {
    rhs = m3_cast (src_t, EXPR_REF (-1));
  }
  if (o == 0 && TREE_TYPE (v) == dst_t) {
    lhs = v;
  } else {
    lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),
       bitsize_int (o));
  }
  if (src_t != dst_t) {
    rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs);
  }
  expand_assignment (lhs, rhs, 0, 0);
  EXPR_POP ();
}

You should be able to repro this by configuring m3cg for mips64-openbsd.
I've only tested on cygwin host, but I could check others..
 
Thanks,
 - Jay

From jay.krell at cornell.edu  Sat Nov  8 16:45:15 2008
From: jay.krell at cornell.edu (Jay)
Date: Sat, 8 Nov 2008 15:45:15 +0000
Subject: [M3devel] FW: m3_load/m3_store/bit_field_ref lose offset??
Message-ID: <COL101-W3DB0BA622C4D31A82B602E6180@phx.gbl>


truncated as usual...along with losing a lot of whitespace..
Let's see...


Tony, how about switching these #if's?
I explain some of why below.

static void
m3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T)
{
  tree val;
#if 0
  if (o != 0 || TREE_TYPE (v) != dst_t) {
    v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),
     bitsize_int (o));
  }
#else
  /* failsafe, but inefficient */
  if (o != 0 || TREE_TYPE (v) != dst_t) {
    v = m3_build1 (ADDR_EXPR, t_addr, v);
    v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));
    v = m3_build1 (INDIRECT_REF, dst_t,
     m3_cast (build_pointer_type (dst_t), v));
  }
#endif
  TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */
  val = m3_cast (src_t, EXPR_REF (-1));
  if (src_T != dst_T) {
    val = m3_build1 (CONVERT_EXPR, dst_t, val);
  }
  add_stmt (build2 (MODIFY_EXPR, dst_t, v, val));
  EXPR_POP ();
}
m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T)
{
#if 0
  if (o != 0 || TREE_TYPE (v) != src_t) {
    v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),
     bitsize_int (o));
  }
#else
  /* failsafe, but inefficient */
  if (o != 0 || TREE_TYPE (v) != src_t) {
    v = m3_build1 (ADDR_EXPR, t_addr, v);
    v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));
    v = m3_build1 (INDIRECT_REF, src_t,
     m3_cast (build_pointer_type (src_t), v));
  }
#endif
  TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */
  if (src_T != dst_T) {
    v = m3_build1 (CONVERT_EXPR, dst_t, v);
  }
  EXPR_PUSH (v);
}

I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).
In an early version it seems that many loads/stores of global variables is wrong.
The "first" code affected is RTLinker_ExpandModuleTable.

PROCEDURE ExpandModuleTable () =
  CONST InitialTableSize = 500;
  VAR new_mods: ADDRESS;  n_bytes: INTEGER;
  BEGIN
    IF (modules = NIL) THEN
      (* first time... *)
      max_modules := InitialTableSize;
      modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr));
      IF (modules = NIL) THEN Cstdlib.abort (); END;
    ELSE
      n_bytes := max_modules * BYTESIZE (RT0.ModulePtr);
      new_mods := Cstdlib.malloc (n_bytes + n_bytes);
      IF (new_mods = NIL) THEN Cstdlib.abort (); END;
      EVAL Cstring.memcpy (new_mods, modules, n_bytes);
      Cstdlib.free (modules);
      modules := new_mods;
      INC (max_modules, max_modules);
    END;
  END ExpandModuleTable;

It should read back modules as NIL.
What happens is it reads back modules as not NIL,
and then max_modules should also be zero, but is "large", leading to malloc to fail,
and abort called.

The output of cm3cg -y is reasonable.
It indicates offset loads.

What is incorrect is that in many loads and stores, but not all, the
offsets are lost.

(923) begin_procedure
  procedure RTLinker__ExpandModuleTable
 RTLinker__ExpandModuleTable(924) set_source_line
  source line  208
(925) load_nil
(926) store
  store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb
(927) set_source_line
  source line  207
(928) set_source_line
  source line  210
(929) load_nil
(930) load
  m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xb

Here is the incorrect code:

 .globl RTLinker__ExpandModuleTable
 .stabd 46,0,0
 .stabn 68,0,206,.LM108
.LM108:
.LFBB8:
 .set nomips16
 .ent RTLinker__ExpandModuleTable
RTLinker__ExpandModuleTable:
 .frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0
 .mask 0xd0000000,-8
 .fmask 0x00000000,0
 daddiu $sp,$sp,-64
 sd $31,56($sp)
 sd $fp,48($sp)
 sd $28,40($sp)
 move $fp,$sp
 lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
 daddu $28,$28,$25
 daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
.LBB9:
 .stabn 68,0,208,.LM109
.LM109:
 sd $0,16($fp)
 .stabn 68,0,210,.LM110
.LM110:
 dla $2,MM_RTLinker
 ld $2,0($2)      << WRONG
 bne $2,$0,.L61
 .stabn 68,0,212,.LM111
.LM111:
 dla $3,MM_RTLinker
 ld $2,128($3)
 andi $2,$2,0x0
 ori $2,$2,0x1f4
 sd $2,128($3)
 .stabn 68,0,213,.LM112
.LM112:
 li $4,4000   # 0xfa0
 jal malloc
 sd $2,0($fp)
 ld $4,0($fp)
 dla $3,MM_RTLinker
 ld $2,136($3)
 andi $2,$2,0x0
 or $2,$4,$2
 sd $2,136($3)     << OK, offset seems low, but it agrees with elsewhere 
 .stabn 68,0,214,.LM113
.LM113:
 dla $2,MM_RTLinker
 ld $2,0($2)       << WRONG 
 bne $2,$0,.L65
 jal abort
 b .L65
.L61:
 .stabn 68,0,216,.LM114
.LM114:
 dla $2,MM_RTLinker
 ld $2,0($2)        << WRONG 
 dsll $2,$2,3
 sd $2,8($fp)
 .stabn 68,0,217,.LM115
.LM115:
 ld $3,8($fp)
 ld $2,8($fp)
 daddu $2,$3,$2
 move $4,$2
 jal malloc
 sd $2,0($fp)
 ld $2,0($fp)
 sd $2,16($fp)
 .stabn 68,0,218,.LM116
.LM116:
 ld $2,16($fp)
 bne $2,$0,.L64
 jal abort
.L64:
 .stabn 68,0,219,.LM117
.LM117:
 ld $4,16($fp)
 dla $2,MM_RTLinker
 ld $2,0($2)          << WRONG
 ld $3,8($fp)
 move $5,$2
 move $6,$3
 jal memcpy
 sd $2,0($fp)
 .stabn 68,0,220,.LM118
.LM118:
 dla $2,MM_RTLinker
 ld $2,0($2)           << WRONG
 move $4,$2
 jal free
 .stabn 68,0,221,.LM119
.LM119:
 ld $4,16($fp)
 dla $3,MM_RTLinker
 ld $2,136($3)         << OK
 andi $2,$2,0x0
 or $2,$4,$2
 sd $2,136($3)         << OK 
 .stabn 68,0,222,.LM120
.LM120:
 dla $2,MM_RTLinker
 ld $3,0($2)           << WRONG
 dla $2,MM_RTLinker
 ld $2,0($2)           << WRONG
 daddu $4,$3,$2
 dla $3,MM_RTLinker
 ld $2,128($3)         << OK
 andi $2,$2,0x0
 or $2,$2,$4
 sd $2,128($3)         << OK
.L65:
.LBE9:
 .stabn 68,0,224,.LM121
.LM121:
 move $sp,$fp
 ld $31,56($sp)
 ld $fp,48($sp)
 ld $28,40($sp)
 daddiu $sp,$sp,64
 j $31
 .end RTLinker__ExpandModuleTable

Here is corrected code:

RTLinker__ExpandModuleTable:
 .frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0
 .mask 0xd0000000,-8
 .fmask 0x00000000,0
 daddiu $sp,$sp,-64
 sd $31,56($sp)
 sd $fp,48($sp)
 sd $28,40($sp)
 move $fp,$sp
 lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
 daddu $28,$28,$25
 daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
.LBB9:
 .stabn 68,0,208,.LM109
.LM109:
 sd $0,16($fp)
 .stabn 68,0,210,.LM110
.LM110:
 dla $2,MM_RTLinker
 daddiu $2,$2,136     << CORRECT (offset seems low, but at least it isn't zero) 
 ld $2,0($2)
 bne $2,$0,.L61
 .stabn 68,0,212,.LM111
.LM111:
 dla $2,MM_RTLinker
 daddiu $3,$2,128     << CORRECT 
 li $2,500   # 0x1f4
 sd $2,0($3)
 .stabn 68,0,213,.LM112
.LM112:
 li $4,4000   # 0xfa0
 jal malloc
 sd $2,0($fp)
 dla $2,MM_RTLinker
 daddiu $3,$2,136      << CORRECT 
 ld $2,0($fp)
 sd $2,0($3)
 .stabn 68,0,214,.LM113
.LM113:
 dla $2,MM_RTLinker
 daddiu $2,$2,136      << CORRECT 
 ld $2,0($2)
 bne $2,$0,.L65
 jal abort
 b .L65
.L61:

I still dump core with this change, but I get much further.
I have not tested this change at all otherwise -- i.e. on working platforms.

I have not debugged through the gcc/m3cg code to see why
the offsets are only sometimes lost, or why other platforms
have no problem. Is MIPS's "global pointer" unusual these days?
I know SPARC has similar, and all the non-x86, non-AMD64 NT platforms
had similar, but that's essentially nothing now (I'm not counting IA64).
I know Linux has PLT (procedure linkage table) and GOT (global offset table).
Same thing?
I haven't looked much at other MIPS ports yet.

http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8

is what made it look about like it does -- in particularing using BIT_FIELD_REF.

1.7:

static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */
     tree v;
     int o;
     tree src_t, dest_t;
     m3_type src_T, dest_T;
{
  if (o == 0 && TREE_TYPE (v) == src_t) {
    EXPR_PUSH (v);
  } else {
    tree adr = m3_build1 (ADDR_EXPR, t_addr, v);
    if (o != 0) {
      adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT));
    }
    EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t,
                          m3_cast (build_pointer_type (src_t), adr)));
  }
#if 1
  if (src_T != dest_T) {
    EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1));
  }
  if (debug_vars) {
    const char *name = "noname";
    if (v != 0 && DECL_NAME(v) != 0) {
      name = IDENTIFIER_POINTER(DECL_NAME(v));
    }
    fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n", name,
            o, src_T, dest_T);
  }
#else
  if (src_T != dest_T) {
    if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) {
      EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1));
    } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) {
      EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1));
    } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) {
      /* okay */
    } else {
      fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n",
                  src_T, dest_T);
    }
  }
#endif
}
static void
m3_store (v, o, src_t, dest_t)  /* GCC32OK */
     tree v;
     int o;
     tree src_t, dest_t;
{
  tree lhs, rhs;
  if (TREE_TYPE (EXPR_REF (-1)) == src_t) {
    rhs = EXPR_REF (-1);
  } else {
    rhs = m3_cast (src_t, EXPR_REF (-1));
  }
  if (o == 0 && TREE_TYPE (v) == dest_t) {
    lhs = v;
  } else {
    tree f = make_node (FIELD_DECL);
    TREE_TYPE (f) = dest_t;
    DECL_ALIGN (f) = TYPE_ALIGN (dest_t);
    DECL_SIZE (f) = TYPE_SIZE (dest_t);
    DECL_MODE (f) = TYPE_MODE (dest_t);
    DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT);
    DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT);
    DECL_FIELD_CONTEXT (f) = TREE_TYPE (v);
    lhs = m3_build2 (COMPONENT_REF, dest_t, v, f);
  }
#if 1
    expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);
#else
  if (dest_t ==  NULL) {
    expand_assignment (lhs, rhs, 0, 0);
  } else {
    expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);
  }
#endif
  EXPR_POP ();
}

1.8:
static void
m3_load (tree v, int o,
  tree src_t, m3_type src_T,
  tree dst_t, m3_type dst_T) /* GCC32OK */
{
  if (o == 0 && TREE_TYPE (v) == src_t) {
    EXPR_PUSH (v);
  } else {
    EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),
     bitsize_int (o)));
  }
  if (src_T != dst_T) {
    EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1));
  }
  if (option_vars_trace) {
    const char *name = "noname";
    if (v != 0 && DECL_NAME(v) != 0) {
      name = IDENTIFIER_POINTER(DECL_NAME(v));
    }
    fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n", name,
     o, src_T, dst_T);
  }
}
static void
m3_store (tree v, int o, tree src_t, tree dst_t)  /* GCC32OK */
{
  tree lhs, rhs;
  if (TREE_TYPE (EXPR_REF (-1)) == src_t) {
    rhs = EXPR_REF (-1);
  } else {
    rhs = m3_cast (src_t, EXPR_REF (-1));
  }
  if (o == 0 && TREE_TYPE (v) == dst_t) {
    lhs = v;
  } else {
    lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),
       bitsize_int (o));
  }
  if (src_t != dst_t) {
    rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs);
  }
  expand_assignment (lhs, rhs, 0, 0);
  EXPR_POP ();
}

 - Jay

From hosking at cs.purdue.edu  Sat Nov  8 23:55:54 2008
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Sat, 8 Nov 2008 22:55:54 +0000
Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
In-Reply-To: <COL101-W3DB0BA622C4D31A82B602E6180@phx.gbl>
References: <COL101-W3DB0BA622C4D31A82B602E6180@phx.gbl>
Message-ID: <C47F6B5A-F154-4485-B839-96185B59E3B6@cs.purdue.edu>

Have you checked to make sure that the pre-gcc IR makes sense?  Does  
it see these values as 32-bit or 64-bit?  The reason I ask is that  
there may be something wrong in your m3middle target specs that cause  
bad IR.  Once we know the IR looks right then we can try and fix the  
problem.

Antony Hosking | Associate Professor | Computer Science | Purdue  
University
305 N. University Street | West Lafayette | IN 47907 | USA
Office +1 765 494 6001 | Mobile +1 765 427 5484



On 8 Nov 2008, at 15:45, Jay wrote:

>
> truncated as usual...along with losing a lot of whitespace..
> Let's see...
>
>
> Tony, how about switching these #if's?
> I explain some of why below.
>
> static void
> m3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t,  
> m3_type dst_T)
> {
>  tree val;
> #if 0
>  if (o != 0 || TREE_TYPE (v) != dst_t) {
>    v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),
>     bitsize_int (o));
>  }
> #else
>  /* failsafe, but inefficient */
>  if (o != 0 || TREE_TYPE (v) != dst_t) {
>    v = m3_build1 (ADDR_EXPR, t_addr, v);
>    v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));
>    v = m3_build1 (INDIRECT_REF, dst_t,
>     m3_cast (build_pointer_type (dst_t), v));
>  }
> #endif
>  TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing  
> problems */
>  val = m3_cast (src_t, EXPR_REF (-1));
>  if (src_T != dst_T) {
>    val = m3_build1 (CONVERT_EXPR, dst_t, val);
>  }
>  add_stmt (build2 (MODIFY_EXPR, dst_t, v, val));
>  EXPR_POP ();
> }
> m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t,  
> m3_type dst_T)
> {
> #if 0
>  if (o != 0 || TREE_TYPE (v) != src_t) {
>    v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),
>     bitsize_int (o));
>  }
> #else
>  /* failsafe, but inefficient */
>  if (o != 0 || TREE_TYPE (v) != src_t) {
>    v = m3_build1 (ADDR_EXPR, t_addr, v);
>    v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));
>    v = m3_build1 (INDIRECT_REF, src_t,
>     m3_cast (build_pointer_type (src_t), v));
>  }
> #endif
>  TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing  
> problems */
>  if (src_T != dst_T) {
>    v = m3_build1 (CONVERT_EXPR, dst_t, v);
>  }
>  EXPR_PUSH (v);
> }
>
> I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).
> In an early version it seems that many loads/stores of global  
> variables is wrong.
> The "first" code affected is RTLinker_ExpandModuleTable.
>
> PROCEDURE ExpandModuleTable () =
>  CONST InitialTableSize = 500;
>  VAR new_mods: ADDRESS;  n_bytes: INTEGER;
>  BEGIN
>    IF (modules = NIL) THEN
>      (* first time... *)
>      max_modules := InitialTableSize;
>      modules := Cstdlib.malloc (InitialTableSize * BYTESIZE  
> (RT0.ModulePtr));
>      IF (modules = NIL) THEN Cstdlib.abort (); END;
>    ELSE
>      n_bytes := max_modules * BYTESIZE (RT0.ModulePtr);
>      new_mods := Cstdlib.malloc (n_bytes + n_bytes);
>      IF (new_mods = NIL) THEN Cstdlib.abort (); END;
>      EVAL Cstring.memcpy (new_mods, modules, n_bytes);
>      Cstdlib.free (modules);
>      modules := new_mods;
>      INC (max_modules, max_modules);
>    END;
>  END ExpandModuleTable;
>
> It should read back modules as NIL.
> What happens is it reads back modules as not NIL,
> and then max_modules should also be zero, but is "large", leading to  
> malloc to fail,
> and abort called.
>
> The output of cm3cg -y is reasonable.
> It indicates offset loads.
>
> What is incorrect is that in many loads and stores, but not all, the
> offsets are lost.
>
> (923) begin_procedure
>  procedure RTLinker__ExpandModuleTable
> RTLinker__ExpandModuleTable(924) set_source_line
>  source line  208
> (925) load_nil
> (926) store
>  store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb
> (927) set_source_line
>  source line  207
> (928) set_source_line
>  source line  210
> (929) load_nil
> (930) load
>  m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xb
>
> Here is the incorrect code:
>
> .globl RTLinker__ExpandModuleTable
> .stabd 46,0,0
> .stabn 68,0,206,.LM108
> .LM108:
> .LFBB8:
> .set nomips16
> .ent RTLinker__ExpandModuleTable
> RTLinker__ExpandModuleTable:
> .frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0
> .mask 0xd0000000,-8
> .fmask 0x00000000,0
> daddiu $sp,$sp,-64
> sd $31,56($sp)
> sd $fp,48($sp)
> sd $28,40($sp)
> move $fp,$sp
> lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
> daddu $28,$28,$25
> daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
> .LBB9:
> .stabn 68,0,208,.LM109
> .LM109:
> sd $0,16($fp)
> .stabn 68,0,210,.LM110
> .LM110:
> dla $2,MM_RTLinker
> ld $2,0($2)      << WRONG
> bne $2,$0,.L61
> .stabn 68,0,212,.LM111
> .LM111:
> dla $3,MM_RTLinker
> ld $2,128($3)
> andi $2,$2,0x0
> ori $2,$2,0x1f4
> sd $2,128($3)
> .stabn 68,0,213,.LM112
> .LM112:
> li $4,4000   # 0xfa0
> jal malloc
> sd $2,0($fp)
> ld $4,0($fp)
> dla $3,MM_RTLinker
> ld $2,136($3)
> andi $2,$2,0x0
> or $2,$4,$2
> sd $2,136($3)     << OK, offset seems low, but it agrees with  
> elsewhere
> .stabn 68,0,214,.LM113
> .LM113:
> dla $2,MM_RTLinker
> ld $2,0($2)       << WRONG
> bne $2,$0,.L65
> jal abort
> b .L65
> .L61:
> .stabn 68,0,216,.LM114
> .LM114:
> dla $2,MM_RTLinker
> ld $2,0($2)        << WRONG
> dsll $2,$2,3
> sd $2,8($fp)
> .stabn 68,0,217,.LM115
> .LM115:
> ld $3,8($fp)
> ld $2,8($fp)
> daddu $2,$3,$2
> move $4,$2
> jal malloc
> sd $2,0($fp)
> ld $2,0($fp)
> sd $2,16($fp)
> .stabn 68,0,218,.LM116
> .LM116:
> ld $2,16($fp)
> bne $2,$0,.L64
> jal abort
> .L64:
> .stabn 68,0,219,.LM117
> .LM117:
> ld $4,16($fp)
> dla $2,MM_RTLinker
> ld $2,0($2)          << WRONG
> ld $3,8($fp)
> move $5,$2
> move $6,$3
> jal memcpy
> sd $2,0($fp)
> .stabn 68,0,220,.LM118
> .LM118:
> dla $2,MM_RTLinker
> ld $2,0($2)           << WRONG
> move $4,$2
> jal free
> .stabn 68,0,221,.LM119
> .LM119:
> ld $4,16($fp)
> dla $3,MM_RTLinker
> ld $2,136($3)         << OK
> andi $2,$2,0x0
> or $2,$4,$2
> sd $2,136($3)         << OK
> .stabn 68,0,222,.LM120
> .LM120:
> dla $2,MM_RTLinker
> ld $3,0($2)           << WRONG
> dla $2,MM_RTLinker
> ld $2,0($2)           << WRONG
> daddu $4,$3,$2
> dla $3,MM_RTLinker
> ld $2,128($3)         << OK
> andi $2,$2,0x0
> or $2,$2,$4
> sd $2,128($3)         << OK
> .L65:
> .LBE9:
> .stabn 68,0,224,.LM121
> .LM121:
> move $sp,$fp
> ld $31,56($sp)
> ld $fp,48($sp)
> ld $28,40($sp)
> daddiu $sp,$sp,64
> j $31
> .end RTLinker__ExpandModuleTable
>
> Here is corrected code:
>
> RTLinker__ExpandModuleTable:
> .frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0
> .mask 0xd0000000,-8
> .fmask 0x00000000,0
> daddiu $sp,$sp,-64
> sd $31,56($sp)
> sd $fp,48($sp)
> sd $28,40($sp)
> move $fp,$sp
> lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
> daddu $28,$28,$25
> daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
> .LBB9:
> .stabn 68,0,208,.LM109
> .LM109:
> sd $0,16($fp)
> .stabn 68,0,210,.LM110
> .LM110:
> dla $2,MM_RTLinker
> daddiu $2,$2,136     << CORRECT (offset seems low, but at least it  
> isn't zero)
> ld $2,0($2)
> bne $2,$0,.L61
> .stabn 68,0,212,.LM111
> .LM111:
> dla $2,MM_RTLinker
> daddiu $3,$2,128     << CORRECT
> li $2,500   # 0x1f4
> sd $2,0($3)
> .stabn 68,0,213,.LM112
> .LM112:
> li $4,4000   # 0xfa0
> jal malloc
> sd $2,0($fp)
> dla $2,MM_RTLinker
> daddiu $3,$2,136      << CORRECT
> ld $2,0($fp)
> sd $2,0($3)
> .stabn 68,0,214,.LM113
> .LM113:
> dla $2,MM_RTLinker
> daddiu $2,$2,136      << CORRECT
> ld $2,0($2)
> bne $2,$0,.L65
> jal abort
> b .L65
> .L61:
>
> I still dump core with this change, but I get much further.
> I have not tested this change at all otherwise -- i.e. on working  
> platforms.
>
> I have not debugged through the gcc/m3cg code to see why
> the offsets are only sometimes lost, or why other platforms
> have no problem. Is MIPS's "global pointer" unusual these days?
> I know SPARC has similar, and all the non-x86, non-AMD64 NT platforms
> had similar, but that's essentially nothing now (I'm not counting  
> IA64).
> I know Linux has PLT (procedure linkage table) and GOT (global  
> offset table).
> Same thing?
> I haven't looked much at other MIPS ports yet.
>
> http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8
>
> is what made it look about like it does -- in particularing using  
> BIT_FIELD_REF.
>
> 1.7:
>
> static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */
>     tree v;
>     int o;
>     tree src_t, dest_t;
>     m3_type src_T, dest_T;
> {
>  if (o == 0 && TREE_TYPE (v) == src_t) {
>    EXPR_PUSH (v);
>  } else {
>    tree adr = m3_build1 (ADDR_EXPR, t_addr, v);
>    if (o != 0) {
>      adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o /  
> BITS_PER_UNIT));
>    }
>    EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t,
>                          m3_cast (build_pointer_type (src_t), adr)));
>  }
> #if 1
>  if (src_T != dest_T) {
>    EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1));
>  }
>  if (debug_vars) {
>    const char *name = "noname";
>    if (v != 0 && DECL_NAME(v) != 0) {
>      name = IDENTIFIER_POINTER(DECL_NAME(v));
>    }
>    fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n",  
> name,
>            o, src_T, dest_T);
>  }
> #else
>  if (src_T != dest_T) {
>    if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) {
>      EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1));
>    } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) {
>      EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1));
>    } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) {
>      /* okay */
>    } else {
>      fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n",
>                  src_T, dest_T);
>    }
>  }
> #endif
> }
> static void
> m3_store (v, o, src_t, dest_t)  /* GCC32OK */
>     tree v;
>     int o;
>     tree src_t, dest_t;
> {
>  tree lhs, rhs;
>  if (TREE_TYPE (EXPR_REF (-1)) == src_t) {
>    rhs = EXPR_REF (-1);
>  } else {
>    rhs = m3_cast (src_t, EXPR_REF (-1));
>  }
>  if (o == 0 && TREE_TYPE (v) == dest_t) {
>    lhs = v;
>  } else {
>    tree f = make_node (FIELD_DECL);
>    TREE_TYPE (f) = dest_t;
>    DECL_ALIGN (f) = TYPE_ALIGN (dest_t);
>    DECL_SIZE (f) = TYPE_SIZE (dest_t);
>    DECL_MODE (f) = TYPE_MODE (dest_t);
>    DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT);
>    DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT);
>    DECL_FIELD_CONTEXT (f) = TREE_TYPE (v);
>    lhs = m3_build2 (COMPONENT_REF, dest_t, v, f);
>  }
> #if 1
>    expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0,  
> 0);
> #else
>  if (dest_t ==  NULL) {
>    expand_assignment (lhs, rhs, 0, 0);
>  } else {
>    expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0,  
> 0);
>  }
> #endif
>  EXPR_POP ();
> }
>
> 1.8:
> static void
> m3_load (tree v, int o,
>  tree src_t, m3_type src_T,
>  tree dst_t, m3_type dst_T) /* GCC32OK */
> {
>  if (o == 0 && TREE_TYPE (v) == src_t) {
>    EXPR_PUSH (v);
>  } else {
>    EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),
>     bitsize_int (o)));
>  }
>  if (src_T != dst_T) {
>    EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1));
>  }
>  if (option_vars_trace) {
>    const char *name = "noname";
>    if (v != 0 && DECL_NAME(v) != 0) {
>      name = IDENTIFIER_POINTER(DECL_NAME(v));
>    }
>    fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n",  
> name,
>     o, src_T, dst_T);
>  }
> }
> static void
> m3_store (tree v, int o, tree src_t, tree dst_t)  /* GCC32OK */
> {
>  tree lhs, rhs;
>  if (TREE_TYPE (EXPR_REF (-1)) == src_t) {
>    rhs = EXPR_REF (-1);
>  } else {
>    rhs = m3_cast (src_t, EXPR_REF (-1));
>  }
>  if (o == 0 && TREE_TYPE (v) == dst_t) {
>    lhs = v;
>  } else {
>    lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),
>       bitsize_int (o));
>  }
>  if (src_t != dst_t) {
>    rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs);
>  }
>  expand_assignment (lhs, rhs, 0, 0);
>  EXPR_POP ();
> }
>
> - Jay

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081108/4bd834d6/attachment-0002.html>

From jay.krell at cornell.edu  Sun Nov  9 02:25:32 2008
From: jay.krell at cornell.edu (Jay)
Date: Sun, 9 Nov 2008 01:25:32 +0000
Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
In-Reply-To: <C47F6B5A-F154-4485-B839-96185B59E3B6@cs.purdue.edu>
References: <COL101-W3DB0BA622C4D31A82B602E6180@phx.gbl>
	<C47F6B5A-F154-4485-B839-96185B59E3B6@cs.purdue.edu>
Message-ID: <COL101-W2CFF8666D1486BD514159E61B0@phx.gbl>


Thanks for the tip/direction.
 
Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the same" but for two systematic diffs:
 
 - AMD64_LINUX has a bunch of added calls to check_nil.
   This is a "bug" in it, inoptimal but ok code, because "first readable address" is 0. It should probably be 4k. I should write a program that iterates starting at 0 to find the value.
  (or better yet, cm3 should have a -configure option that does this? -- for configuration values for which there is a safe default; in general, imho, the targets should be auto configured via C, assuming there is a native development environment available).
 
 - some integers are different (again, systematically) 
 
AMD64_LINUX:
 init_int  656 13 Int.64 init_chars  664 "    typecode " init_int  680 2 Int.64 init_var  688 v.1 0 init_int  696 13 Int.64 init_chars  704 "    typeid   " init_int  720 2 Int.64 init_var  728 v.1 0 init_int  736 13 Int.64 init_chars  744 "RunMainBody: " init_int  760 2 Int.64 init_var  768 v.1 0 init_int  776 19 Int.64 init_chars  784 "RunMainBody: exec: " init_int  808 2 Int.64 init_var  816 v.1 0 init_int  824 17 Int.64 init_chars  832 "ExpandInitStack: "
MIPS64_OPENBSD:
 
 init_chars  664 "    typecode " init_int  680 8796093022208 Int.64 init_var  688 v.1 0 init_int  696 13 Int.64 init_chars  704 "    typeid   " init_int  720 8796093022208 Int.64 init_var  728 v.1 0 init_int  736 13 Int.64 init_chars  744 "RunMainBody: " init_int  760 8796093022208 Int.64 init_var  768 v.1 0 init_int  776 19 Int.64 init_chars  784 "RunMainBody: exec: " init_int  808 8796093022208 Int.64 init_var  816 v.1 0This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where little endian has 2.
 
I don't know what these numbers represent.
I should look at other systems -- preferably a working 64 bit big endian system, if there are any.
 
Could be flags/type with text literals?
 
(Btw, I always leave the TextLiteral.i3 change on my system, lowering the maximum literal length to around 4gig even on 64 bit targets, so 32 bit can bootstrap 64 bit...need to fix the compiler here to do target arithmetic instead of host arithmetic...)
 
Thanks, - Jay



From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sat, 8 Nov 2008 22:55:54 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Have you checked to make sure that the pre-gcc IR makes sense?  Does it see these values as 32-bit or 64-bit?  The reason I ask is that there may be something wrong in your m3middle target specs that cause bad IR.  Once we know the IR looks right then we can try and fix the problem.



Antony Hosking | Associate Professor | Computer Science | Purdue University
305 N. University Street | West Lafayette | IN 47907 | USA
Office +1 765 494 6001 | Mobile +1 765 427 5484


On 8 Nov 2008, at 15:45, Jay wrote:

truncated as usual...along with losing a lot of whitespace..Let's see...Tony, how about switching these #if's?I explain some of why below.static voidm3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){ tree val;#if 0 if (o != 0 || TREE_TYPE (v) != dst_t) {   v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),    bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) {   v = m3_build1 (ADDR_EXPR, t_addr, v);   v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));   v = m3_build1 (INDIRECT_REF, dst_t,    m3_cast (build_pointer_type (dst_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) {   val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP ();}m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){#if 0 if (o != 0 || TREE_TYPE (v) != src_t) {   v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),    bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) {   v = m3_build1 (ADDR_EXPR, t_addr, v);   v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));   v = m3_build1 (INDIRECT_REF, src_t,    m3_cast (build_pointer_type (src_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) {   v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v);}I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).In an early version it seems that many loads/stores of global variables is wrong.The "first" code affected is RTLinker_ExpandModuleTable.PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS;  n_bytes: INTEGER; BEGIN   IF (modules = NIL) THEN     (* first time... *)     max_modules := InitialTableSize;     modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr));     IF (modules = NIL) THEN Cstdlib.abort (); END;   ELSE     n_bytes := max_modules * BYTESIZE (RT0.ModulePtr);     new_mods := Cstdlib.malloc (n_bytes + n_bytes);     IF (new_mods = NIL) THEN Cstdlib.abort (); END;     EVAL Cstring.memcpy (new_mods, modules, n_bytes);     Cstdlib.free (modules);     modules := new_mods;     INC (max_modules, max_modules);   END; END ExpandModuleTable;It should read back modules as NIL.What happens is it reads back modules as not NIL,and then max_modules should also be zero, but is "large", leading to malloc to fail,and abort called.The output of cm3cg -y is reasonable.It indicates offset loads.What is incorrect is that in many loads and stores, but not all, theoffsets are lost.(923) begin_procedure procedure RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable(924) set_source_line source line  208(925) load_nil(926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb(927) set_source_line source line  207(928) set_source_line source line  210(929) load_nil(930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xbHere is the incorrect code:.globl RTLinker__ExpandModuleTable.stabd 46,0,0.stabn 68,0,206,.LM108.LM108:.LFBB8:.set nomips16.ent RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable:.frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerld $2,0($2)      << WRONGbne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $3,MM_RTLinkerld $2,128($3)andi $2,$2,0x0ori $2,$2,0x1f4sd $2,128($3).stabn 68,0,213,.LM112.LM112:li $4,4000   # 0xfa0jal mallocsd $2,0($fp)ld $4,0($fp)dla $3,MM_RTLinkerld $2,136($3)andi $2,$2,0x0or $2,$4,$2sd $2,136($3)     << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerld $2,0($2)       << WRONG bne $2,$0,.L65jal abortb .L65.L61:.stabn 68,0,216,.LM114.LM114:dla $2,MM_RTLinkerld $2,0($2)        << WRONG dsll $2,$2,3sd $2,8($fp).stabn 68,0,217,.LM115.LM115:ld $3,8($fp)ld $2,8($fp)daddu $2,$3,$2move $4,$2jal mallocsd $2,0($fp)ld $2,0($fp)sd $2,16($fp).stabn 68,0,218,.LM116.LM116:ld $2,16($fp)bne $2,$0,.L64jal abort.L64:.stabn 68,0,219,.LM117.LM117:ld $4,16($fp)dla $2,MM_RTLinkerld $2,0($2)          << WRONGld $3,8($fp)move $5,$2move $6,$3jal memcpysd $2,0($fp).stabn 68,0,220,.LM118.LM118:dla $2,MM_RTLinkerld $2,0($2)           << WRONGmove $4,$2jal free.stabn 68,0,221,.LM119.LM119:ld $4,16($fp)dla $3,MM_RTLinkerld $2,136($3)         << OKandi $2,$2,0x0or $2,$4,$2sd $2,136($3)         << OK .stabn 68,0,222,.LM120.LM120:dla $2,MM_RTLinkerld $3,0($2)           << WRONGdla $2,MM_RTLinkerld $2,0($2)           << WRONGdaddu $4,$3,$2dla $3,MM_RTLinkerld $2,128($3)         << OKandi $2,$2,0x0or $2,$2,$4sd $2,128($3)         << OK.L65:.LBE9:.stabn 68,0,224,.LM121.LM121:move $sp,$fpld $31,56($sp)ld $fp,48($sp)ld $28,40($sp)daddiu $sp,$sp,64j $31.end RTLinker__ExpandModuleTableHere is corrected code:RTLinker__ExpandModuleTable:.frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerdaddiu $2,$2,136     << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2)bne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $2,MM_RTLinkerdaddiu $3,$2,128     << CORRECT li $2,500   # 0x1f4sd $2,0($3).stabn 68,0,213,.LM112.LM112:li $4,4000   # 0xfa0jal mallocsd $2,0($fp)dla $2,MM_RTLinkerdaddiu $3,$2,136      << CORRECT ld $2,0($fp)sd $2,0($3).stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerdaddiu $2,$2,136      << CORRECT ld $2,0($2)bne $2,$0,.L65jal abortb .L65.L61:I still dump core with this change, but I get much further.I have not tested this change at all otherwise -- i.e. on working platforms.I have not debugged through the gcc/m3cg code to see whythe offsets are only sometimes lost, or why other platformshave no problem. Is MIPS's "global pointer" unusual these days?I know SPARC has similar, and all the non-x86, non-AMD64 NT platformshad similar, but that's essentially nothing now (I'm not counting IA64).I know Linux has PLT (procedure linkage table) and GOT (global offset table).Same thing?I haven't looked much at other MIPS ports yet.http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8is what made it look about like it does -- in particularing using BIT_FIELD_REF.1.7:static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */    tree v;    int o;    tree src_t, dest_t;    m3_type src_T, dest_T;{ if (o == 0 && TREE_TYPE (v) == src_t) {   EXPR_PUSH (v); } else {   tree adr = m3_build1 (ADDR_EXPR, t_addr, v);   if (o != 0) {     adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT));   }   EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t,                         m3_cast (build_pointer_type (src_t), adr))); }#if 1 if (src_T != dest_T) {   EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) {   const char *name = "noname";   if (v != 0 && DECL_NAME(v) != 0) {     name = IDENTIFIER_POINTER(DECL_NAME(v));   }   fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n", name,           o, src_T, dest_T); }#else if (src_T != dest_T) {   if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) {     EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1));   } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) {     EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1));   } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) {     /* okay */   } else {     fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n",                 src_T, dest_T);   } }#endif}static voidm3_store (v, o, src_t, dest_t)  /* GCC32OK */    tree v;    int o;    tree src_t, dest_t;{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) {   rhs = EXPR_REF (-1); } else {   rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) {   lhs = v; } else {   tree f = make_node (FIELD_DECL);   TREE_TYPE (f) = dest_t;   DECL_ALIGN (f) = TYPE_ALIGN (dest_t);   DECL_SIZE (f) = TYPE_SIZE (dest_t);   DECL_MODE (f) = TYPE_MODE (dest_t);   DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT);   DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT);   DECL_FIELD_CONTEXT (f) = TREE_TYPE (v);   lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); }#if 1   expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);#else if (dest_t ==  NULL) {   expand_assignment (lhs, rhs, 0, 0); } else {   expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); }#endif EXPR_POP ();}1.8:static voidm3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */{ if (o == 0 && TREE_TYPE (v) == src_t) {   EXPR_PUSH (v); } else {   EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),    bitsize_int (o))); } if (src_T != dst_T) {   EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) {   const char *name = "noname";   if (v != 0 && DECL_NAME(v) != 0) {     name = IDENTIFIER_POINTER(DECL_NAME(v));   }   fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n", name,    o, src_T, dst_T); }}static voidm3_store (tree v, int o, tree src_t, tree dst_t)  /* GCC32OK */{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) {   rhs = EXPR_REF (-1); } else {   rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) {   lhs = v; } else {   lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),      bitsize_int (o)); } if (src_t != dst_t) {   rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP ();}- Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081109/2528e9ca/attachment-0002.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: AMD64_LINUX_RTLinker.mc
Type: application/octet-stream
Size: 17622 bytes
Desc: not available
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081109/2528e9ca/attachment-0004.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: MIPS64_OPENBSD_RTLinker.mc
Type: application/octet-stream
Size: 17692 bytes
Desc: not available
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081109/2528e9ca/attachment-0005.obj>

From hosking at cs.purdue.edu  Sun Nov  9 11:01:58 2008
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Sun, 9 Nov 2008 10:01:58 +0000
Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
In-Reply-To: <COL101-W2CFF8666D1486BD514159E61B0@phx.gbl>
References: <COL101-W3DB0BA622C4D31A82B602E6180@phx.gbl>
	<C47F6B5A-F154-4485-B839-96185B59E3B6@cs.purdue.edu>
	<COL101-W2CFF8666D1486BD514159E61B0@phx.gbl>
Message-ID: <B9A95C1E-B873-40A9-A045-CD38E32CA5D0@cs.purdue.edu>

Aren't the integer values supposed to be written endian-independently  
in the IR?  I am suspicious about these differences.

On 9 Nov 2008, at 01:25, Jay wrote:

> Thanks for the tip/direction.
>
> Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the  
> same" but for two systematic diffs:
>
>  - AMD64_LINUX has a bunch of added calls to check_nil.
>    This is a "bug" in it, inoptimal but ok code, because "first  
> readable address" is 0. It should probably be 4k. I should write a  
> program that iterates starting at 0 to find the value.
>   (or better yet, cm3 should have a -configure option that does  
> this? -- for configuration values for which there is a safe default;  
> in general, imho, the targets should be auto configured via C,  
> assuming there is a native development environment available).
>
>  - some integers are different (again, systematically)
>
> AMD64_LINUX:
>  init_int  656 13 Int.64
>  init_chars  664 "    typecode "
>  init_int  680 2 Int.64
>  init_var  688 v.1 0
>  init_int  696 13 Int.64
>  init_chars  704 "    typeid   "
>  init_int  720 2 Int.64
>  init_var  728 v.1 0
>  init_int  736 13 Int.64
>  init_chars  744 "RunMainBody: "
>  init_int  760 2 Int.64
>  init_var  768 v.1 0
>  init_int  776 19 Int.64
>  init_chars  784 "RunMainBody: exec: "
>  init_int  808 2 Int.64
>  init_var  816 v.1 0
>  init_int  824 17 Int.64
>  init_chars  832 "ExpandInitStack: "
>
> MIPS64_OPENBSD:
>
>  init_chars  664 "    typecode "
>  init_int  680 8796093022208 Int.64
>  init_var  688 v.1 0
>  init_int  696 13 Int.64
>  init_chars  704 "    typeid   "
>  init_int  720 8796093022208 Int.64
>  init_var  728 v.1 0
>  init_int  736 13 Int.64
>  init_chars  744 "RunMainBody: "
>  init_int  760 8796093022208 Int.64
>  init_var  768 v.1 0
>  init_int  776 19 Int.64
>  init_chars  784 "RunMainBody: exec: "
>  init_int  808 8796093022208 Int.64
>  init_var  816 v.1 0
>
> This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where  
> little endian has 2.
>
> I don't know what these numbers represent.
> I should look at other systems -- preferably a working 64 bit big  
> endian system, if there are any.
>
> Could be flags/type with text literals?
>
> (Btw, I always leave the TextLiteral.i3 change on my system,  
> lowering the maximum literal length to around 4gig even on 64 bit  
> targets, so 32 bit can bootstrap 64 bit...need to fix the compiler  
> here to do target arithmetic instead of host arithmetic...)
>
> Thanks,
>  - Jay
>
>
> From: hosking at cs.purdue.edu
> To: jay.krell at cornell.edu
> Date: Sat, 8 Nov 2008 22:55:54 +0000
> CC: m3devel at elegosoft.com
> Subject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
>
> Have you checked to make sure that the pre-gcc IR makes sense?  Does  
> it see these values as 32-bit or 64-bit?  The reason I ask is that  
> there may be something wrong in your m3middle target specs that  
> cause bad IR.  Once we know the IR looks right then we can try and  
> fix the problem.
>
> Antony Hosking | Associate Professor | Computer Science | Purdue  
> University
> 305 N. University Street | West Lafayette | IN 47907 | USA
> Office +1 765 494 6001 | Mobile +1 765 427 5484
>
>
>
> On 8 Nov 2008, at 15:45, Jay wrote:
>
>
> truncated as usual...along with losing a lot of whitespace..
> Let's see...
>
>
> Tony, how about switching these #if's?
> I explain some of why below.
>
> static void
> m3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t,  
> m3_type dst_T)
> {
>  tree val;
> #if 0
>  if (o != 0 || TREE_TYPE (v) != dst_t) {
>    v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),
>     bitsize_int (o));
>  }
> #else
>  /* failsafe, but inefficient */
>  if (o != 0 || TREE_TYPE (v) != dst_t) {
>    v = m3_build1 (ADDR_EXPR, t_addr, v);
>    v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));
>    v = m3_build1 (INDIRECT_REF, dst_t,
>     m3_cast (build_pointer_type (dst_t), v));
>  }
> #endif
>  TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing  
> problems */
>  val = m3_cast (src_t, EXPR_REF (-1));
>  if (src_T != dst_T) {
>    val = m3_build1 (CONVERT_EXPR, dst_t, val);
>  }
>  add_stmt (build2 (MODIFY_EXPR, dst_t, v, val));
>  EXPR_POP ();
> }
> m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t,  
> m3_type dst_T)
> {
> #if 0
>  if (o != 0 || TREE_TYPE (v) != src_t) {
>    v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),
>     bitsize_int (o));
>  }
> #else
>  /* failsafe, but inefficient */
>  if (o != 0 || TREE_TYPE (v) != src_t) {
>    v = m3_build1 (ADDR_EXPR, t_addr, v);
>    v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));
>    v = m3_build1 (INDIRECT_REF, src_t,
>     m3_cast (build_pointer_type (src_t), v));
>  }
> #endif
>  TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing  
> problems */
>  if (src_T != dst_T) {
>    v = m3_build1 (CONVERT_EXPR, dst_t, v);
>  }
>  EXPR_PUSH (v);
> }
>
> I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).
> In an early version it seems that many loads/stores of global  
> variables is wrong.
> The "first" code affected is RTLinker_ExpandModuleTable.
>
> PROCEDURE ExpandModuleTable () =
>  CONST InitialTableSize = 500;
>  VAR new_mods: ADDRESS;  n_bytes: INTEGER;
>  BEGIN
>    IF (modules = NIL) THEN
>      (* first time... *)
>      max_modules := InitialTableSize;
>      modules := Cstdlib.malloc (InitialTableSize * BYTESIZE  
> (RT0.ModulePtr));
>      IF (modules = NIL) THEN Cstdlib.abort (); END;
>    ELSE
>      n_bytes := max_modules * BYTESIZE (RT0.ModulePtr);
>      new_mods := Cstdlib.malloc (n_bytes + n_bytes);
>      IF (new_mods = NIL) THEN Cstdlib.abort (); END;
>      EVAL Cstring.memcpy (new_mods, modules, n_bytes);
>      Cstdlib.free (modules);
>      modules := new_mods;
>      INC (max_modules, max_modules);
>    END;
>  END ExpandModuleTable;
>
> It should read back modules as NIL.
> What happens is it reads back modules as not NIL,
> and then max_modules should also be zero, but is "large", leading to  
> malloc to fail,
> and abort called.
>
> The output of cm3cg -y is reasonable.
> It indicates offset loads.
>
> What is incorrect is that in many loads and stores, but not all, the
> offsets are lost.
>
> (923) begin_procedure
>  procedure RTLinker__ExpandModuleTable
> RTLinker__ExpandModuleTable(924) set_source_line
>  source line  208
> (925) load_nil
> (926) store
>  store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb
> (927) set_source_line
>  source line  207
> (928) set_source_line
>  source line  210
> (929) load_nil
> (930) load
>  m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xb
>
> Here is the incorrect code:
>
> .globl RTLinker__ExpandModuleTable
> .stabd 46,0,0
> .stabn 68,0,206,.LM108
> .LM108:
> .LFBB8:
> .set nomips16
> .ent RTLinker__ExpandModuleTable
> RTLinker__ExpandModuleTable:
> .frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0
> .mask 0xd0000000,-8
> .fmask 0x00000000,0
> daddiu $sp,$sp,-64
> sd $31,56($sp)
> sd $fp,48($sp)
> sd $28,40($sp)
> move $fp,$sp
> lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
> daddu $28,$28,$25
> daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
> .LBB9:
> .stabn 68,0,208,.LM109
> .LM109:
> sd $0,16($fp)
> .stabn 68,0,210,.LM110
> .LM110:
> dla $2,MM_RTLinker
> ld $2,0($2)      << WRONG
> bne $2,$0,.L61
> .stabn 68,0,212,.LM111
> .LM111:
> dla $3,MM_RTLinker
> ld $2,128($3)
> andi $2,$2,0x0
> ori $2,$2,0x1f4
> sd $2,128($3)
> .stabn 68,0,213,.LM112
> .LM112:
> li $4,4000   # 0xfa0
> jal malloc
> sd $2,0($fp)
> ld $4,0($fp)
> dla $3,MM_RTLinker
> ld $2,136($3)
> andi $2,$2,0x0
> or $2,$4,$2
> sd $2,136($3)     << OK, offset seems low, but it agrees with  
> elsewhere
> .stabn 68,0,214,.LM113
> .LM113:
> dla $2,MM_RTLinker
> ld $2,0($2)       << WRONG
> bne $2,$0,.L65
> jal abort
> b .L65
> .L61:
> .stabn 68,0,216,.LM114
> .LM114:
> dla $2,MM_RTLinker
> ld $2,0($2)        << WRONG
> dsll $2,$2,3
> sd $2,8($fp)
> .stabn 68,0,217,.LM115
> .LM115:
> ld $3,8($fp)
> ld $2,8($fp)
> daddu $2,$3,$2
> move $4,$2
> jal malloc
> sd $2,0($fp)
> ld $2,0($fp)
> sd $2,16($fp)
> .stabn 68,0,218,.LM116
> .LM116:
> ld $2,16($fp)
> bne $2,$0,.L64
> jal abort
> .L64:
> .stabn 68,0,219,.LM117
> .LM117:
> ld $4,16($fp)
> dla $2,MM_RTLinker
> ld $2,0($2)          << WRONG
> ld $3,8($fp)
> move $5,$2
> move $6,$3
> jal memcpy
> sd $2,0($fp)
> .stabn 68,0,220,.LM118
> .LM118:
> dla $2,MM_RTLinker
> ld $2,0($2)           << WRONG
> move $4,$2
> jal free
> .stabn 68,0,221,.LM119
> .LM119:
> ld $4,16($fp)
> dla $3,MM_RTLinker
> ld $2,136($3)         << OK
> andi $2,$2,0x0
> or $2,$4,$2
> sd $2,136($3)         << OK
> .stabn 68,0,222,.LM120
> .LM120:
> dla $2,MM_RTLinker
> ld $3,0($2)           << WRONG
> dla $2,MM_RTLinker
> ld $2,0($2)           << WRONG
> daddu $4,$3,$2
> dla $3,MM_RTLinker
> ld $2,128($3)         << OK
> andi $2,$2,0x0
> or $2,$2,$4
> sd $2,128($3)         << OK
> .L65:
> .LBE9:
> .stabn 68,0,224,.LM121
> .LM121:
> move $sp,$fp
> ld $31,56($sp)
> ld $fp,48($sp)
> ld $28,40($sp)
> daddiu $sp,$sp,64
> j $31
> .end RTLinker__ExpandModuleTable
>
> Here is corrected code:
>
> RTLinker__ExpandModuleTable:
> .frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0
> .mask 0xd0000000,-8
> .fmask 0x00000000,0
> daddiu $sp,$sp,-64
> sd $31,56($sp)
> sd $fp,48($sp)
> sd $28,40($sp)
> move $fp,$sp
> lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
> daddu $28,$28,$25
> daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
> .LBB9:
> .stabn 68,0,208,.LM109
> .LM109:
> sd $0,16($fp)
> .stabn 68,0,210,.LM110
> .LM110:
> dla $2,MM_RTLinker
> daddiu $2,$2,136     << CORRECT (offset seems low, but at least it  
> isn't zero)
> ld $2,0($2)
> bne $2,$0,.L61
> .stabn 68,0,212,.LM111
> .LM111:
> dla $2,MM_RTLinker
> daddiu $3,$2,128     << CORRECT
> li $2,500   # 0x1f4
> sd $2,0($3)
> .stabn 68,0,213,.LM112
> .LM112:
> li $4,4000   # 0xfa0
> jal malloc
> sd $2,0($fp)
> dla $2,MM_RTLinker
> daddiu $3,$2,136      << CORRECT
> ld $2,0($fp)
> sd $2,0($3)
> .stabn 68,0,214,.LM113
> .LM113:
> dla $2,MM_RTLinker
> daddiu $2,$2,136      << CORRECT
> ld $2,0($2)
> bne $2,$0,.L65
> jal abort
> b .L65
> .L61:
>
> I still dump core with this change, but I get much further.
> I have not tested this change at all otherwise -- i.e. on working  
> platforms.
>
> I have not debugged through the gcc/m3cg code to see why
> the offsets are only sometimes lost, or why other platforms
> have no problem. Is MIPS's "global pointer" unusual these days?
> I know SPARC has similar, and all the non-x86, non-AMD64 NT platforms
> had similar, but that's essentially nothing now (I'm not counting  
> IA64).
> I know Linux has PLT (procedure linkage table) and GOT (global  
> offset table).
> Same thing?
> I haven't looked much at other MIPS ports yet.
>
> http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8
>
> is what made it look about like it does -- in particularing using  
> BIT_FIELD_REF.
>
> 1.7:
>
> static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */
>     tree v;
>     int o;
>     tree src_t, dest_t;
>     m3_type src_T, dest_T;
> {
>  if (o == 0 && TREE_TYPE (v) == src_t) {
>    EXPR_PUSH (v);
>  } else {
>    tree adr = m3_build1 (ADDR_EXPR, t_addr, v);
>    if (o != 0) {
>      adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o /  
> BITS_PER_UNIT));
>    }
>    EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t,
>                          m3_cast (build_pointer_type (src_t), adr)));
>  }
> #if 1
>  if (src_T != dest_T) {
>    EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1));
>  }
>  if (debug_vars) {
>    const char *name = "noname";
>    if (v != 0 && DECL_NAME(v) != 0) {
>      name = IDENTIFIER_POINTER(DECL_NAME(v));
>    }
>    fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n",  
> name,
>            o, src_T, dest_T);
>  }
> #else
>  if (src_T != dest_T) {
>    if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) {
>      EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1));
>    } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) {
>      EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1));
>    } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) {
>      /* okay */
>    } else {
>      fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n",
>                  src_T, dest_T);
>    }
>  }
> #endif
> }
> static void
> m3_store (v, o, src_t, dest_t)  /* GCC32OK */
>     tree v;
>     int o;
>     tree src_t, dest_t;
> {
>  tree lhs, rhs;
>  if (TREE_TYPE (EXPR_REF (-1)) == src_t) {
>    rhs = EXPR_REF (-1);
>  } else {
>    rhs = m3_cast (src_t, EXPR_REF (-1));
>  }
>  if (o == 0 && TREE_TYPE (v) == dest_t) {
>    lhs = v;
>  } else {
>    tree f = make_node (FIELD_DECL);
>    TREE_TYPE (f) = dest_t;
>    DECL_ALIGN (f) = TYPE_ALIGN (dest_t);
>    DECL_SIZE (f) = TYPE_SIZE (dest_t);
>    DECL_MODE (f) = TYPE_MODE (dest_t);
>    DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT);
>    DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT);
>    DECL_FIELD_CONTEXT (f) = TREE_TYPE (v);
>    lhs = m3_build2 (COMPONENT_REF, dest_t, v, f);
>  }
> #if 1
>    expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0,  
> 0);
> #else
>  if (dest_t ==  NULL) {
>    expand_assignment (lhs, rhs, 0, 0);
>  } else {
>    expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0,  
> 0);
>  }
> #endif
>  EXPR_POP ();
> }
>
> 1.8:
> static void
> m3_load (tree v, int o,
>  tree src_t, m3_type src_T,
>  tree dst_t, m3_type dst_T) /* GCC32OK */
> {
>  if (o == 0 && TREE_TYPE (v) == src_t) {
>    EXPR_PUSH (v);
>  } else {
>    EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),
>     bitsize_int (o)));
>  }
>  if (src_T != dst_T) {
>    EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1));
>  }
>  if (option_vars_trace) {
>    const char *name = "noname";
>    if (v != 0 && DECL_NAME(v) != 0) {
>      name = IDENTIFIER_POINTER(DECL_NAME(v));
>    }
>    fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n",  
> name,
>     o, src_T, dst_T);
>  }
> }
> static void
> m3_store (tree v, int o, tree src_t, tree dst_t)  /* GCC32OK */
> {
>  tree lhs, rhs;
>  if (TREE_TYPE (EXPR_REF (-1)) == src_t) {
>    rhs = EXPR_REF (-1);
>  } else {
>    rhs = m3_cast (src_t, EXPR_REF (-1));
>  }
>  if (o == 0 && TREE_TYPE (v) == dst_t) {
>    lhs = v;
>  } else {
>    lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),
>       bitsize_int (o));
>  }
>  if (src_t != dst_t) {
>    rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs);
>  }
>  expand_assignment (lhs, rhs, 0, 0);
>  EXPR_POP ();
> }
>
> - Jay
>
> <AMD64_LINUX_RTLinker.mc><MIPS64_OPENBSD_RTLinker.mc>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081109/b8730591/attachment-0002.html>

From jay.krell at cornell.edu  Sun Nov  9 12:10:54 2008
From: jay.krell at cornell.edu (Jay)
Date: Sun, 9 Nov 2008 11:10:54 +0000
Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
In-Reply-To: <B9A95C1E-B873-40A9-A045-CD38E32CA5D0@cs.purdue.edu>
References: <COL101-W3DB0BA622C4D31A82B602E6180@phx.gbl>
	<C47F6B5A-F154-4485-B839-96185B59E3B6@cs.purdue.edu>
	<COL101-W2CFF8666D1486BD514159E61B0@phx.gbl> 
	<B9A95C1E-B873-40A9-A045-CD38E32CA5D0@cs.purdue.edu>
Message-ID: <BAY123-W4494232D2A947D3041E17AE61B0@phx.gbl>


I agree. I am suspicious too. I will investigate more. Thanks.
 
 - Jay



From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 9 Nov 2008 10:01:58 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
Aren't the integer values supposed to be written endian-independently in the IR?  I am suspicious about these differences.




On 9 Nov 2008, at 01:25, Jay wrote:

Thanks for the tip/direction. Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the same" but for two systematic diffs:  - AMD64_LINUX has a bunch of added calls to check_nil.   This is a "bug" in it, inoptimal but ok code, because "first readable address" is 0. It should probably be 4k. I should write a program that iterates starting at 0 to find the value.  (or better yet, cm3 should have a -configure option that does this? -- for configuration values for which there is a safe default; in general, imho, the targets should be auto configured via C, assuming there is a native development environment available).  - some integers are different (again, systematically)  AMD64_LINUX: init_int  656 13 Int.64 init_chars  664 "    typecode " init_int  680 2 Int.64 init_var  688 v.1 0 init_int  696 13 Int.64 init_chars  704 "    typeid   " init_int  720 2 Int.64 init_var  728 v.1 0 init_int  736 13 Int.64 init_chars  744 "RunMainBody: " init_int  760 2 Int.64 init_var  768 v.1 0 init_int  776 19 Int.64 init_chars  784 "RunMainBody: exec: " init_int  808 2 Int.64 init_var  816 v.1 0 init_int  824 17 Int.64 init_chars  832 "ExpandInitStack: "MIPS64_OPENBSD:  init_chars  664 "    typecode " init_int  680 8796093022208 Int.64 init_var  688 v.1 0 init_int  696 13 Int.64 init_chars  704 "    typeid   " init_int  720 8796093022208 Int.64 init_var  728 v.1 0 init_int  736 13 Int.64 init_chars  744 "RunMainBody: " init_int  760 8796093022208 Int.64 init_var  768 v.1 0 init_int  776 19 Int.64 init_chars  784 "RunMainBody: exec: " init_int  808 8796093022208 Int.64 init_var  816 v.1 0This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where little endian has 2. I don't know what these numbers represent.I should look at other systems -- preferably a working 64 bit big endian system, if there are any. Could be flags/type with text literals? (Btw, I always leave the TextLiteral.i3 change on my system, lowering the maximum literal length to around 4gig even on 64 bit targets, so 32 bit can bootstrap 64 bit...need to fix the compiler here to do target arithmetic instead of host arithmetic...) Thanks, - Jay

From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sat, 8 Nov 2008 22:55:54 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Have you checked to make sure that the pre-gcc IR makes sense?  Does it see these values as 32-bit or 64-bit?  The reason I ask is that there may be something wrong in your m3middle target specs that cause bad IR.  Once we know the IR looks right then we can try and fix the problem.



Antony Hosking | Associate Professor | Computer Science | Purdue University
305 N. University Street | West Lafayette | IN 47907 | USA
Office +1 765 494 6001 | Mobile +1 765 427 5484


On 8 Nov 2008, at 15:45, Jay wrote:

truncated as usual...along with losing a lot of whitespace..Let's see...Tony, how about switching these #if's?I explain some of why below.static voidm3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){ tree val;#if 0 if (o != 0 || TREE_TYPE (v) != dst_t) {   v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),    bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) {   v = m3_build1 (ADDR_EXPR, t_addr, v);   v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));   v = m3_build1 (INDIRECT_REF, dst_t,    m3_cast (build_pointer_type (dst_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) {   val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP ();}m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){#if 0 if (o != 0 || TREE_TYPE (v) != src_t) {   v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),    bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) {   v = m3_build1 (ADDR_EXPR, t_addr, v);   v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));   v = m3_build1 (INDIRECT_REF, src_t,    m3_cast (build_pointer_type (src_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) {   v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v);}I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).In an early version it seems that many loads/stores of global variables is wrong.The "first" code affected is RTLinker_ExpandModuleTable.PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS;  n_bytes: INTEGER; BEGIN   IF (modules = NIL) THEN     (* first time... *)     max_modules := InitialTableSize;     modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr));     IF (modules = NIL) THEN Cstdlib.abort (); END;   ELSE     n_bytes := max_modules * BYTESIZE (RT0.ModulePtr);     new_mods := Cstdlib.malloc (n_bytes + n_bytes);     IF (new_mods = NIL) THEN Cstdlib.abort (); END;     EVAL Cstring.memcpy (new_mods, modules, n_bytes);     Cstdlib.free (modules);     modules := new_mods;     INC (max_modules, max_modules);   END; END ExpandModuleTable;It should read back modules as NIL.What happens is it reads back modules as not NIL,and then max_modules should also be zero, but is "large", leading to malloc to fail,and abort called.The output of cm3cg -y is reasonable.It indicates offset loads.What is incorrect is that in many loads and stores, but not all, theoffsets are lost.(923) begin_procedure procedure RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable(924) set_source_line source line  208(925) load_nil(926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb(927) set_source_line source line  207(928) set_source_line source line  210(929) load_nil(930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xbHere is the incorrect code:.globl RTLinker__ExpandModuleTable.stabd 46,0,0.stabn 68,0,206,.LM108.LM108:.LFBB8:.set nomips16.ent RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable:.frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerld $2,0($2)      << WRONGbne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $3,MM_RTLinkerld $2,128($3)andi $2,$2,0x0ori $2,$2,0x1f4sd $2,128($3).stabn 68,0,213,.LM112.LM112:li $4,4000   # 0xfa0jal mallocsd $2,0($fp)ld $4,0($fp)dla $3,MM_RTLinkerld $2,136($3)andi $2,$2,0x0or $2,$4,$2sd $2,136($3)     << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerld $2,0($2)       << WRONG bne $2,$0,.L65jal abortb .L65.L61:.stabn 68,0,216,.LM114.LM114:dla $2,MM_RTLinkerld $2,0($2)        << WRONG dsll $2,$2,3sd $2,8($fp).stabn 68,0,217,.LM115.LM115:ld $3,8($fp)ld $2,8($fp)daddu $2,$3,$2move $4,$2jal mallocsd $2,0($fp)ld $2,0($fp)sd $2,16($fp).stabn 68,0,218,.LM116.LM116:ld $2,16($fp)bne $2,$0,.L64jal abort.L64:.stabn 68,0,219,.LM117.LM117:ld $4,16($fp)dla $2,MM_RTLinkerld $2,0($2)          << WRONGld $3,8($fp)move $5,$2move $6,$3jal memcpysd $2,0($fp).stabn 68,0,220,.LM118.LM118:dla $2,MM_RTLinkerld $2,0($2)           << WRONGmove $4,$2jal free.stabn 68,0,221,.LM119.LM119:ld $4,16($fp)dla $3,MM_RTLinkerld $2,136($3)         << OKandi $2,$2,0x0or $2,$4,$2sd $2,136($3)         << OK .stabn 68,0,222,.LM120.LM120:dla $2,MM_RTLinkerld $3,0($2)           << WRONGdla $2,MM_RTLinkerld $2,0($2)           << WRONGdaddu $4,$3,$2dla $3,MM_RTLinkerld $2,128($3)         << OKandi $2,$2,0x0or $2,$2,$4sd $2,128($3)         << OK.L65:.LBE9:.stabn 68,0,224,.LM121.LM121:move $sp,$fpld $31,56($sp)ld $fp,48($sp)ld $28,40($sp)daddiu $sp,$sp,64j $31.end RTLinker__ExpandModuleTableHere is corrected code:RTLinker__ExpandModuleTable:.frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerdaddiu $2,$2,136     << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2)bne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $2,MM_RTLinkerdaddiu $3,$2,128     << CORRECT li $2,500   # 0x1f4sd $2,0($3).stabn 68,0,213,.LM112.LM112:li $4,4000   # 0xfa0jal mallocsd $2,0($fp)dla $2,MM_RTLinkerdaddiu $3,$2,136      << CORRECT ld $2,0($fp)sd $2,0($3).stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerdaddiu $2,$2,136      << CORRECT ld $2,0($2)bne $2,$0,.L65jal abortb .L65.L61:I still dump core with this change, but I get much further.I have not tested this change at all otherwise -- i.e. on working platforms.I have not debugged through the gcc/m3cg code to see whythe offsets are only sometimes lost, or why other platformshave no problem. Is MIPS's "global pointer" unusual these days?I know SPARC has similar, and all the non-x86, non-AMD64 NT platformshad similar, but that's essentially nothing now (I'm not counting IA64).I know Linux has PLT (procedure linkage table) and GOT (global offset table).Same thing?I haven't looked much at other MIPS ports yet.http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8is what made it look about like it does -- in particularing using BIT_FIELD_REF.1.7:static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */    tree v;    int o;    tree src_t, dest_t;    m3_type src_T, dest_T;{ if (o == 0 && TREE_TYPE (v) == src_t) {   EXPR_PUSH (v); } else {   tree adr = m3_build1 (ADDR_EXPR, t_addr, v);   if (o != 0) {     adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT));   }   EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t,                         m3_cast (build_pointer_type (src_t), adr))); }#if 1 if (src_T != dest_T) {   EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) {   const char *name = "noname";   if (v != 0 && DECL_NAME(v) != 0) {     name = IDENTIFIER_POINTER(DECL_NAME(v));   }   fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n", name,           o, src_T, dest_T); }#else if (src_T != dest_T) {   if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) {     EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1));   } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) {     EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1));   } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) {     /* okay */   } else {     fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n",                 src_T, dest_T);   } }#endif}static voidm3_store (v, o, src_t, dest_t)  /* GCC32OK */    tree v;    int o;    tree src_t, dest_t;{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) {   rhs = EXPR_REF (-1); } else {   rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) {   lhs = v; } else {   tree f = make_node (FIELD_DECL);   TREE_TYPE (f) = dest_t;   DECL_ALIGN (f) = TYPE_ALIGN (dest_t);   DECL_SIZE (f) = TYPE_SIZE (dest_t);   DECL_MODE (f) = TYPE_MODE (dest_t);   DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT);   DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT);   DECL_FIELD_CONTEXT (f) = TREE_TYPE (v);   lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); }#if 1   expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);#else if (dest_t ==  NULL) {   expand_assignment (lhs, rhs, 0, 0); } else {   expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); }#endif EXPR_POP ();}1.8:static voidm3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */{ if (o == 0 && TREE_TYPE (v) == src_t) {   EXPR_PUSH (v); } else {   EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),    bitsize_int (o))); } if (src_T != dst_T) {   EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) {   const char *name = "noname";   if (v != 0 && DECL_NAME(v) != 0) {     name = IDENTIFIER_POINTER(DECL_NAME(v));   }   fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n", name,    o, src_T, dst_T); }}static voidm3_store (tree v, int o, tree src_t, tree dst_t)  /* GCC32OK */{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) {   rhs = EXPR_REF (-1); } else {   rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) {   lhs = v; } else {   lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),      bitsize_int (o)); } if (src_t != dst_t) {   rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP ();}- Jay<AMD64_LINUX_RTLinker.mc><MIPS64_OPENBSD_RTLinker.mc>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081109/f520ce42/attachment-0002.html>

From jay.krell at cornell.edu  Sun Nov  9 15:09:10 2008
From: jay.krell at cornell.edu (Jay)
Date: Sun, 9 Nov 2008 14:09:10 +0000
Subject: [M3devel] AMD64_LINUX gc/stubgen
Message-ID: <COL101-W14DAC6D23849CBD1BE010AE61B0@phx.gbl>


Tony, any tips on this one? :)
 
jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 -sno RemoteView.T -T.M3IMPTAB @M3nogc
 
always succeeds.
 
jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 -sno RemoteView.T   -T.M3IMPTAB
 
usually fails:
****** runtime error:***    NEW() was unable to allocate more memory.***    file "../src/runtime/common/RTAllocator.m3", line 285***Formatter.m3:
 
CONST ChunkSize = 128; (* Must be a power of 2 *)
PROCEDURE New(wr: Wr.T;  width: CARDINAL:= 75): T RAISES {} =  BEGIN    WITH t = NEW(T) DO      t.wr := wr;      t.width := width;      t.indent := 0;        t.numChars := 0;        (* t.stream := RefStream{NIL, ..}; *)        t.buffer := NEW (ExprBuf, 4 * ChunkSize);  << this is the line that fails
 
Lowering chunk size to 64 seemed to make it succeed more often, but not always.
I think I'll go back to MIPS64_OPENBSD..probably easier to track that problem down.
 
 - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081109/00d5c623/attachment-0002.html>

From hosking at cs.purdue.edu  Sun Nov  9 19:02:41 2008
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Sun, 9 Nov 2008 18:02:41 +0000
Subject: [M3devel] AMD64_LINUX gc/stubgen
In-Reply-To: <COL101-W14DAC6D23849CBD1BE010AE61B0@phx.gbl>
References: <COL101-W14DAC6D23849CBD1BE010AE61B0@phx.gbl>
Message-ID: <97806969-D1C7-4D77-8B54-466088B99937@cs.purdue.edu>

Clearly a problem with GC.  I'm guessing you have something wrong  
(pointer alignment?) in the configuration.  I suspect if you run with  
@M3paranoidgc you'll trip over the problem.

On 9 Nov 2008, at 14:09, Jay wrote:

> Tony, any tips on this one? :)
>
> jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 - 
> sno RemoteView.T -T.M3IMPTAB @M3nogc
>
> always succeeds.
>
> jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 - 
> sno RemoteView.T   -T.M3IMPTAB
>
> usually fails:
>
> ***
> *** runtime error:
> ***    NEW() was unable to allocate more memory.
> ***    file "../src/runtime/common/RTAllocator.m3", line 285
> ***
>
> Formatter.m3:
>
> CONST ChunkSize = 128; (* Must be a power of 2 *)
>
> PROCEDURE New(wr: Wr.T;  width: CARDINAL:= 75): T RAISES {} =
>   BEGIN
>     WITH t = NEW(T) DO
>       t.wr := wr;
>       t.width := width;
>       t.indent := 0;
>
>       t.numChars := 0;
>
>       (* t.stream := RefStream{NIL, ..}; *)
>
>       t.buffer := NEW (ExprBuf, 4 * ChunkSize);  << this is the line  
> that fails
>
> Lowering chunk size to 64 seemed to make it succeed more often, but  
> not always.
> I think I'll go back to MIPS64_OPENBSD..probably easier to track  
> that problem down.
>
>  - Jay

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081109/623733d0/attachment-0002.html>

From jay.krell at cornell.edu  Mon Nov 10 09:28:18 2008
From: jay.krell at cornell.edu (Jay)
Date: Mon, 10 Nov 2008 08:28:18 +0000
Subject: [M3devel] AMD64_LINUX gc/stubgen
In-Reply-To: <97806969-D1C7-4D77-8B54-466088B99937@cs.purdue.edu>
References: <COL101-W14DAC6D23849CBD1BE010AE61B0@phx.gbl>
	<97806969-D1C7-4D77-8B54-466088B99937@cs.purdue.edu>
Message-ID: <COL101-W13EBD249838CF77D46F8D2E61A0@phx.gbl>


@M3paranoidgc always succeeds. I see, reading the code, it only does reads. Nevertheless.
 
Simply programs that just allocate memory forever also succeed.
Such as this:
 
jay at amd64a:/dev2/cm3/t$ cat src/t.m3MODULE t EXPORTS Main;
TYPE T1 = RECORD c : ARRAY [0..10000000] OF INTEGER; END;TYPE T2 = RECORD c : ARRAY [0..4095] OF INTEGER; END;
 
VAR a: REFANY;
 
PROCEDURE F1()=VAR b: REFANY;
BEGIN  WHILE TRUE DO    b := NEW(REF T2);    a := NEW(REF T1);  END;END F1;
 
BEGIN  F1();END t.
 
Darn.
 
 - Jay



From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 9 Nov 2008 18:02:41 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] AMD64_LINUX gc/stubgen
Clearly a problem with GC.  I'm guessing you have something wrong (pointer alignment?) in the configuration.  I suspect if you run with @M3paranoidgc you'll trip over the problem.




On 9 Nov 2008, at 14:09, Jay wrote:

Tony, any tips on this one? :) jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 -sno RemoteView.T -T.M3IMPTAB @M3nogc always succeeds. jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 -sno RemoteView.T   -T.M3IMPTAB usually fails:****** runtime error:***    NEW() was unable to allocate more memory.***    file "../src/runtime/common/RTAllocator.m3", line 285***Formatter.m3: CONST ChunkSize = 128; (* Must be a power of 2 *)PROCEDURE New(wr: Wr.T;  width: CARDINAL:= 75): T RAISES {} =  BEGIN    WITH t = NEW(T) DO      t.wr := wr;      t.width := width;      t.indent := 0;        t.numChars := 0;        (* t.stream := RefStream{NIL, ..}; *)        t.buffer := NEW (ExprBuf, 4 * ChunkSize);  << this is the line that fails Lowering chunk size to 64 seemed to make it succeed more often, but not always.I think I'll go back to MIPS64_OPENBSD..probably easier to track that problem down.  - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081110/da2e695b/attachment-0002.html>

From hosking at cs.purdue.edu  Mon Nov 10 12:13:05 2008
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Mon, 10 Nov 2008 11:13:05 +0000
Subject: [M3devel] AMD64_LINUX gc/stubgen
In-Reply-To: <COL101-W13EBD249838CF77D46F8D2E61A0@phx.gbl>
References: <COL101-W14DAC6D23849CBD1BE010AE61B0@phx.gbl>
	<97806969-D1C7-4D77-8B54-466088B99937@cs.purdue.edu>
	<COL101-W13EBD249838CF77D46F8D2E61A0@phx.gbl>
Message-ID: <831F958E-6D14-4DDD-AEF1-7BF3AAA3BC1B@cs.purdue.edu>

Hmm.  Can you try @M3paranoidgc @M3noincremental @M3nogenerational?

On 10 Nov 2008, at 08:28, Jay wrote:

> @M3paranoidgc always succeeds. I see, reading the code, it only does  
> reads. Nevertheless.
>
> Simply programs that just allocate memory forever also succeed.
> Such as this:
>
> jay at amd64a:/dev2/cm3/t$ cat src/t.m3
> MODULE t EXPORTS Main;
> TYPE T1 = RECORD c : ARRAY [0..10000000] OF INTEGER; END;
> TYPE T2 = RECORD c : ARRAY [0..4095] OF INTEGER; END;
>
> VAR a: REFANY;
>
> PROCEDURE F1()=
> VAR b: REFANY;
> BEGIN
>   WHILE TRUE DO
>     b := NEW(REF T2);
>     a := NEW(REF T1);
>   END;
> END F1;
>
> BEGIN
>   F1();
> END t.
>
> Darn.
>
>  - Jay
>
>
> From: hosking at cs.purdue.edu
> To: jay.krell at cornell.edu
> Date: Sun, 9 Nov 2008 18:02:41 +0000
> CC: m3devel at elegosoft.com
> Subject: Re: [M3devel] AMD64_LINUX gc/stubgen
>
>
> Clearly a problem with GC.  I'm guessing you have something wrong  
> (pointer alignment?) in the configuration.  I suspect if you run  
> with @M3paranoidgc you'll trip over the problem.
>
> On 9 Nov 2008, at 14:09, Jay wrote:
>
> Tony, any tips on this one? :)
>
> jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 - 
> sno RemoteView.T -T.M3IMPTAB @M3nogc
>
> always succeeds.
>
> jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 - 
> sno RemoteView.T   -T.M3IMPTAB
>
> usually fails:
>
> ***
> *** runtime error:
> ***    NEW() was unable to allocate more memory.
> ***    file "../src/runtime/common/RTAllocator.m3", line 285
> ***
>
> Formatter.m3:
>
> CONST ChunkSize = 128; (* Must be a power of 2 *)
>
> PROCEDURE New(wr: Wr.T;  width: CARDINAL:= 75): T RAISES {} =
>   BEGIN
>     WITH t = NEW(T) DO
>       t.wr := wr;
>       t.width := width;
>       t.indent := 0;
>
>       t.numChars := 0;
>
>       (* t.stream := RefStream{NIL, ..}; *)
>
>       t.buffer := NEW (ExprBuf, 4 * ChunkSize);  << this is the line  
> that fails
>
> Lowering chunk size to 64 seemed to make it succeed more often, but  
> not always.
> I think I'll go back to MIPS64_OPENBSD..probably easier to track  
> that problem down.
>
>  - Jay
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081110/d1ad8649/attachment-0002.html>

From jay.krell at cornell.edu  Mon Nov 10 20:35:19 2008
From: jay.krell at cornell.edu (Jay)
Date: Mon, 10 Nov 2008 19:35:19 +0000
Subject: [M3devel] AMD64_LINUX gc/stubgen
In-Reply-To: <831F958E-6D14-4DDD-AEF1-7BF3AAA3BC1B@cs.purdue.edu>
References: <COL101-W14DAC6D23849CBD1BE010AE61B0@phx.gbl>
	<97806969-D1C7-4D77-8B54-466088B99937@cs.purdue.edu>
	<COL101-W13EBD249838CF77D46F8D2E61A0@phx.gbl> 
	<831F958E-6D14-4DDD-AEF1-7BF3AAA3BC1B@cs.purdue.edu>
Message-ID: <COL101-W35EAC45CFEB850BBEEFDEDE61A0@phx.gbl>


It /seems/ like any switch or switch combination makes it stop happening.
Admittedly I haven't run a loop just a few runs by hand for each combo.
 
Maybe have it record the allocation quantities and "play them back"?
 
 - Jay


From: hosking at cs.purdue.edu
To: jay.krell at cornell.edu
Date: Mon, 10 Nov 2008 11:13:05 +0000
CC: m3devel at elegosoft.com
Subject: Re: [M3devel] AMD64_LINUX gc/stubgen




Hmm.  Can you try @M3paranoidgc @M3noincremental @M3nogenerational?


On 10 Nov 2008, at 08:28, Jay wrote:


@M3paranoidgc always succeeds. I see, reading the code, it only does reads. Nevertheless.
 
Simply programs that just allocate memory forever also succeed.
Such as this:
 
jay at amd64a:/dev2/cm3/t$ cat src/t.m3
MODULE t EXPORTS Main;
TYPE T1 = RECORD c : ARRAY [0..10000000] OF INTEGER; END;
TYPE T2 = RECORD c : ARRAY [0..4095] OF INTEGER; END;
 
VAR a: REFANY;
 
PROCEDURE F1()=
VAR b: REFANY;
BEGIN
  WHILE TRUE DO
    b := NEW(REF T2);
    a := NEW(REF T1);
  END;
END F1;
 
BEGIN
  F1();
END t.
 
Darn.
 
 - Jay






From: hosking at cs.purdue.edu
To: jay.krell at cornell.edu
Date: Sun, 9 Nov 2008 18:02:41 +0000
CC: m3devel at elegosoft.com
Subject: Re: [M3devel] AMD64_LINUX gc/stubgen



Clearly a problem with GC.  I'm guessing you have something wrong (pointer alignment?) in the configuration.  I suspect if you run with @M3paranoidgc you'll trip over the problem.





On 9 Nov 2008, at 14:09, Jay wrote:


Tony, any tips on this one? :)
 
jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 -sno RemoteView.T -T.M3IMPTAB @M3nogc
 
always succeeds.
 
jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 -sno RemoteView.T   -T.M3IMPTAB
 
usually fails:

***
*** runtime error:
***    NEW() was unable to allocate more memory.
***    file "../src/runtime/common/RTAllocator.m3", line 285
***

Formatter.m3:
 
CONST ChunkSize = 128; (* Must be a power of 2 *)

PROCEDURE New(wr: Wr.T;  width: CARDINAL:= 75): T RAISES {} =
  BEGIN
    WITH t = NEW(T) DO
      t.wr := wr;
      t.width := width;
      t.indent := 0;
  
      t.numChars := 0;
  
      (* t.stream := RefStream{NIL, ..}; *)
  
      t.buffer := NEW (ExprBuf, 4 * ChunkSize);  << this is the line that fails
 
Lowering chunk size to 64 seemed to make it succeed more often, but not always.
I think I'll go back to MIPS64_OPENBSD..probably easier to track that problem down.
 
 - Jay

From rcoleburn at scires.com  Tue Nov 11 00:17:31 2008
From: rcoleburn at scires.com (Randy Coleburn)
Date: Mon, 10 Nov 2008 18:17:31 -0500
Subject: [M3devel] [M3commit] CVS Update: cm3
In-Reply-To: <20081109062016.407EF10D5894@birch.elegosoft.com>
References: <20081109062016.407EF10D5894@birch.elegosoft.com>
Message-ID: <491879AF.1E75.00D7.1@scires.com>

Jay:
 
Does this mean we will now need to put a "cm3cg.common" file into our bin folder on Windows?  
 
Just checking before I do any updates from the repository so I know what needs to change in my target .
 
Regards,
Randy

>>> Jay Krell <jkrell at elego.de> 11/9/2008 1:26 AM >>>
CVSROOT:/usr/cvs
Changes by:jkrell at birch.08/11/09 07:20:15

Modified files:
cm3/m3-sys/cminstall/src/config/: NT386.common cm3.cfg 
cm3/m3-sys/cminstall/src/config-no-install/: AMD64_LINUX 
                                             LINUXLIBC6 
                                             PPC32_OPENBSD 
                                             PPC_DARWIN 
                                             PPC_LINUX SOLgnu 
                                             SOLsun 
                                             SPARC32_LINUX 
                                             SPARC64_LINUX 
                                             SPARC64_OPENBSD 
                                             Unix.common 
Added files:
cm3/m3-sys/cminstall/src/config-no-install/: MIPS64_OPENBSD 
                                             cm3cfg.common 

Log message:
begin combining nt386.common and unix.common (into cm3cg.common)
mips64_openbsd support
being migrating config-no-install/solgnu and solsun to Unix.common
(had this sitting around, in intermediate state)
go back to old formating because that is all the old C written m3build could handle
add probing for host/target specific cm3cg (so I can stop copying it around
as I switch targets)


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081110/03748963/attachment-0002.html>

From jay.krell at cornell.edu  Tue Nov 11 02:06:07 2008
From: jay.krell at cornell.edu (Jay)
Date: Tue, 11 Nov 2008 01:06:07 +0000
Subject: [M3devel] [M3commit] CVS Update: cm3
In-Reply-To: <491879AF.1E75.00D7.1@scires.com>
References: <20081109062016.407EF10D5894@birch.elegosoft.com>
	<491879AF.1E75.00D7.1@scires.com>
Message-ID: <COL101-W586BB50F95A3C239AE2386E6150@phx.gbl>


If you use my config files, yes.
Note that there are two of them, one just includes the other.
Take "the other".
This is just because on my system, I think:
\cm3\cm3.cfg -- probes around 
 => \dev2\cm3.2\m3-sys\cminstall\src\config\NT386 
 => \dev2\cm3.2\m3-sys\cminstall\src\config\NT386.common  
 => \dev2\cm3.2\m3-sys\cminstall\src\config\cm3cfg.common 
 => \dev2\cm3.2\m3-sys\cminstall\src\config-no-install\cm3cfg.common 
 
NT386 is the oddball for me, since for any other targets, for me, it goes:
 
 \cm3\cm3.cfg -- probes around 
  => \dev2\cm3.2\m3-sys\cminstall\src\config-no-install\platform 
  => \dev2\cm3.2\m3-sys\cminstall\src\config-no-install\Unix.common 
  => \dev2\cm3.2\m3-sys\cminstall\src\config-no-install\cm3cfg.common 
 
I'd like to move NT386* to config-no-install, but I avoided doing so so far.
I'd like to replace config with config-no-install, but avoid that too.
 
 - Jay


Date: Mon, 10 Nov 2008 18:17:31 -0500From: rcoleburn at scires.comTo: m3devel at elegosoft.comSubject: Re: [M3devel] [M3commit] CVS Update: cm3
Jay:
 
Does this mean we will now need to put a "cm3cg.common" file into our bin folder on Windows?  
 
Just checking before I do any updates from the repository so I know what needs to change in my target .
 
Regards,
Randy>>> Jay Krell <jkrell at elego.de> 11/9/2008 1:26 AM >>>CVSROOT:/usr/cvsChanges by:jkrell at birch.08/11/09 07:20:15Modified files:cm3/m3-sys/cminstall/src/config/: NT386.common cm3.cfg cm3/m3-sys/cminstall/src/config-no-install/: AMD64_LINUX                                              LINUXLIBC6                                              PPC32_OPENBSD                                              PPC_DARWIN                                              PPC_LINUX SOLgnu                                              SOLsun                                              SPARC32_LINUX                                              SPARC64_LINUX                                              SPARC64_OPENBSD                                              Unix.common Added files:cm3/m3-sys/cminstall/src/config-no-install/: MIPS64_OPENBSD                                              cm3cfg.common Log message:begin combining nt386.common and unix.common (into cm3cg.common)mips64_openbsd supportbeing migrating config-no-install/solgnu and solsun to Unix.common(had this sitting around, in intermediate state)go back to old formating because that is all the old C written m3build could handleadd probing for host/target specific cm3cg (so I can stop copying it aroundas I switch targets)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081111/4d459506/attachment-0002.html>

From rodney.bates at wichita.edu  Tue Nov 11 21:39:09 2008
From: rodney.bates at wichita.edu (Rodney M. Bates)
Date: Tue, 11 Nov 2008 14:39:09 -0600
Subject: [M3devel] Still can't bootstrap cm3
Message-ID: <4919ED6D.6000700@wichita.edu>

We still have serious problems with the cm3 system building.
I have again gotten into a big hampster wheel of several different
failures, and still can't rebuild a working compiler.
I have gone through enough steps that I am not just positive
I have remembered it all right, but it was something like this:

I started with ./do-cm3-front.sh, ./do-cm3-front.sh -ship, and
./install-cm3-compiler.sh upgrade.  That left me with a compiler
that always failed with:

------------------------------------------------------------------------------------------------------------

rodney at yellowstone:~/proj/m3/cm3-new/cm3/m3-sys/m3middle$ cm3 -build
--- building in LINUXLIBC6 ---

ignoring ../src/m3overrides

"/usr/local/cm3/pkg/libm3/src/bundleintf/bundle-ov.tmpl", line 4: quake runtime 
error: undefined variable: ROOT

--procedure--  -line-  -file---
_import_template       4  /usr/local/cm3/pkg/libm3/src/bundleintf/bundle-ov.tmpl
_import_template       4  /usr/local/cm3/pkg/libm3/src/bundleintf/bundle-ov.tmpl
import            274  /usr/local/cm3/pkg/libm3/LINUXLIBC6/.M3EXPORTS
include_dir        11 
/home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3middle/src/m3makefile
                     4 
/home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3middle/LINUXLIBC6/m3make.args

Fatal Error: package build failed

-----------------------------------------------------------------------------------------------------------

Then I tried rebuilding using various do-cm3-* scripts, including after
supplying argument realclean (which, I have learned many times in the
past, is essential to avoid all sorts of problems), with the same results.
Somewhere in here, I started getting the bad version stamp failures too.
Eventually, trying to rebootstrap with ./upgrade.sh, and also manually,
a package at a time, I was consistently getting:

--------------------------------------------------------------------------------------------------------

rodney at yellowstone:~/proj/m3/cm3-new/cm3/m3-sys/m3middle$ cm3 -build
--- building in LINUXLIBC6 ---

ignoring ../src/m3overrides

"/usr/local/cm3/pkg/libm3/LINUXLIBC6/.M3EXPORTS", line 4: quake runtime error: 
unable to open 
"/home/rodney/proj/m3/cm3-new/cm3/m3-libs/m3core/LINUXLIBC6/.M3EXPORTS" for reading

--procedure--  -line-  -file---
import_version      --  <builtin>
import              4  /usr/local/cm3/pkg/libm3/LINUXLIBC6/.M3EXPORTS
include_dir        11 
/home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3middle/src/m3makefile
                     4 
/home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3middle/LINUXLIBC6/m3make.args

Fatal Error: package build failed

-----------------------------------------------------------------------------------------------------------

Perhaps it was the realclean that deleted all the .M3EXPORTS files.
../boot_cm3-core.sh LINUXLIBC6 gave this failure:

-----------------------------------------------------------------------------------------------------------

rodney at yellowstone:~/proj/m3/cm3-new/cm3/scripts$ ./boot-cm3-core.sh
please specify a cross compilation target platform
rodney at yellowstone:~/proj/m3/cm3-new/cm3/scripts$ ./boot-cm3-core.sh LINUXLIBC6
/home/rodney/proj/m3/cm3-new/cm3/scripts/pkgmap.sh -c "cm3 -build -override 
-DROOT='/home/rodney/proj/m3/cm3-new/cm3' -DCM3_VERSION_TEXT='d5.7.0' 
-DCM3_VERSION_NUMBER='050700' -DCM3_LAST_CHANGED='2008-03-16' 
-DM3_BOOTSTRAP=TRUE -keep" m3core libm3 sysutils m3middle m3objfile m3linker 
m3back m3front m3quake cm3 m3scanner m3tools m3cgcat m3cggen m3bundle bitvector 
digraph parseparams realgeometry set slisp sortedtableextras table-list tempfiles
=== package /home/rodney/proj/m3/cm3-new/cm3/m3-libs/m3core ===
  +++ cm3 -build -override -DROOT='/home/rodney/proj/m3/cm3-new/cm3' 
-DCM3_VERSION_TEXT='d5.7.0' -DCM3_VERSION_NUMBER='050700' 
-DCM3_LAST_CHANGED='2008-03-16' -DM3_BOOTSTRAP=TRUE -keep +++
--- building in LINUXLIBC6 ---

new source -> compiling RTHooks.i3
/bin/sh: 
/home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3cc/LINUXLIBC6-LINUXLIBC6/cm3cg: not found
   m3_backend => 127
m3cc (aka cm3cg) failed compiling: RTHooks.ic

---------------------------------------------------------------------------------------------------------

Fortunately, an earlier ./upgrade.sh had built a cm3cg (though it had failed
on all the Modula-3 code).  I got around the above by putting in a symbolic
link from cm3/m3-sys/m3cc/LINUXLIBC6-LINUXLIBC6 to cm3/m3-sys/m3cc/LINUXLIBC6,
so it could find cm3cg.

Then I went through the package-at-a-time order Tony had given in an old post
to the m3devel list.  That had other problems.  Running cm3 in the package
directory failed with the undefined ROOT problem, but do-pkg.sh would do the
build.  But do-pkg.sh doesn't appear to do a ship, so I had to do that in
each package with cm3 -ship.

After building and shipping m3 middle and m3front, package cm3 itself wouldn't
link, wanting libraries for m3back, m3objfile, m3quake, sysutils, m3linker, libm3,
and m3core.  I skipped it and m3gc*, which don't exist now, and went through the
rest of the order, which completed without failures building.  But that gave
me a compiler that fails with a new problem I have never seen before:

---------------------------------------------------------------------------------------------------------

rodney at yellowstone:~/proj/m3/cm3-new/cm3/scripts$ ./install-cm3-compiler.sh upgrade
cp_if: /usr/local/cm3/bin/cm3 and /usr/local/cm3/bin/cm3-d5.7.0 identical
cp_if: /usr/local/cm3/bin/cm3cg and /usr/local/cm3/bin/cm3cg-d5.7.0 identical


***
*** runtime error:
***    A compile-time type is missing.
***    file "../src/Utils.m3"
***

Aborted (core dumped)
cp /home/rodney/proj/m3/cm3-new/cm3/m3-sys/cm3/LINUXLIBC6/cm3 
/usr/local/cm3/bin/cm3-
cp /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3cc/LINUXLIBC6/cm3cg 
/usr/local/cm3/bin/cm3cg-
cp /usr/local/cm3/bin/cm3- /usr/local/cm3/bin/cm3
cp /usr/local/cm3/bin/cm3cg- /usr/local/cm3/bin/cm3cg

------------------------------------------------------------------------------------------------------------

Going back to an older cm3 makes that go away and makes the ROOT problem
recur.  Working around that with cm3 -DROOT="/home/rodney/proj/m3/cm3-new/cm3"
get the compiler to run, but it doesn't do anything, because it thinks
nothing needs to be recompiled.  I could delete stuff, but I fear that would
just start me around another iteration of the same thing.

I don't believe I have ever gotten a bootstrap to work.  I think I have always
had to download a recent binary and go from there.

Any help would be greatly appreciated.

Rodney Bates



From jay.krell at cornell.edu  Wed Nov 12 05:04:44 2008
From: jay.krell at cornell.edu (Jay)
Date: Wed, 12 Nov 2008 04:04:44 +0000
Subject: [M3devel] Still can't bootstrap cm3
In-Reply-To: <4919ED6D.6000700@wichita.edu>
References: <4919ED6D.6000700@wichita.edu>
Message-ID: <COL101-W81AEB201A820491B221953E6140@phx.gbl>


Try starting from a recent snapshot, not necessarily a recent release, and running upgrade.sh or upgrade.py.

 
ROOT should be set in the config file, or as you showed, on the command line.
upgrade.sh/upgrade.py will figure it out and set it on the command line.
I'd kind of like the compiler to set it itself, but that is kind of inappropriate.


It's definitely possible that starting from too old of a release won't work.

 
Avoid any script with "boot" in the name.
Those are I believe for bringing up new targets and esp. cross building, not for
updating mature working systems that can build natively.
The dash in LINUXLIBC6-LINUXLIBC6 confirms it.
The dash appears when the host and target are different.
The first platform is the host, the second is the target.

 
 - Jay

From wagner at elegosoft.com  Wed Nov 12 08:18:55 2008
From: wagner at elegosoft.com (Olaf Wagner)
Date: Wed, 12 Nov 2008 08:18:55 +0100
Subject: [M3devel] Still can't bootstrap cm3
In-Reply-To: <4919ED6D.6000700@wichita.edu>
References: <4919ED6D.6000700@wichita.edu>
Message-ID: <20081112081855.alr1p7k4ggw0k4k0@mail.elegosoft.com>

Quoting "Rodney M. Bates" <rodney.bates at wichita.edu>:

> We still have serious problems with the cm3 system building.
> I have again gotten into a big hampster wheel of several different
> failures, and still can't rebuild a working compiler.
> I have gone through enough steps that I am not just positive
> I have remembered it all right, but it was something like this:

Rodney,

it's not easy to understand what has gone wrong at this point.
Usually, if you have a working system, cm3/scripts/upgrade.sh
should be able to bootstrap to a new set of sources. This is tested
every day in the tinderbox release tests, too. As the last builds seem
to have succeeded, I don't think there is a real problem in our
code base currently. You may want to have a look at test_build_current
in cm3/scripts/regression/defs.sh to see what's done in the regression
builds.

I'd suggest you either start again from the last release or
one of the latest snapshots available from the build machine.
I assume you are not building a cross compiler, but just trying
to perform an upgrade.

Daily snapshots are available from
http://www.opencm3.net/snaps/snapshot-index.html

I hope this helps,

Olaf

> I started with ./do-cm3-front.sh, ./do-cm3-front.sh -ship, and
> ./install-cm3-compiler.sh upgrade.  That left me with a compiler
> that always failed with:
>
> ------------------------------------------------------------------------------------------------------------
>
> rodney at yellowstone:~/proj/m3/cm3-new/cm3/m3-sys/m3middle$ cm3 -build
> --- building in LINUXLIBC6 ---
>
> ignoring ../src/m3overrides
>
> "/usr/local/cm3/pkg/libm3/src/bundleintf/bundle-ov.tmpl", line 4: quake
> runtime error: undefined variable: ROOT
>
> --procedure--  -line-  -file---
> _import_template       4
> /usr/local/cm3/pkg/libm3/src/bundleintf/bundle-ov.tmpl
> _import_template       4
> /usr/local/cm3/pkg/libm3/src/bundleintf/bundle-ov.tmpl
> import            274  /usr/local/cm3/pkg/libm3/LINUXLIBC6/.M3EXPORTS
> include_dir        11
> /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3middle/src/m3makefile
>                     4
> /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3middle/LINUXLIBC6/m3make.args
>
> Fatal Error: package build failed
>
> -----------------------------------------------------------------------------------------------------------
>
> Then I tried rebuilding using various do-cm3-* scripts, including after
> supplying argument realclean (which, I have learned many times in the
> past, is essential to avoid all sorts of problems), with the same results.
> Somewhere in here, I started getting the bad version stamp failures too.
> Eventually, trying to rebootstrap with ./upgrade.sh, and also manually,
> a package at a time, I was consistently getting:
>
> --------------------------------------------------------------------------------------------------------
>
> rodney at yellowstone:~/proj/m3/cm3-new/cm3/m3-sys/m3middle$ cm3 -build
> --- building in LINUXLIBC6 ---
>
> ignoring ../src/m3overrides
>
> "/usr/local/cm3/pkg/libm3/LINUXLIBC6/.M3EXPORTS", line 4: quake runtime
> error: unable to open
> "/home/rodney/proj/m3/cm3-new/cm3/m3-libs/m3core/LINUXLIBC6/.M3EXPORTS"
> for reading
>
> --procedure--  -line-  -file---
> import_version      --  <builtin>
> import              4  /usr/local/cm3/pkg/libm3/LINUXLIBC6/.M3EXPORTS
> include_dir        11
> /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3middle/src/m3makefile
>                     4
> /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3middle/LINUXLIBC6/m3make.args
>
> Fatal Error: package build failed
>
> -----------------------------------------------------------------------------------------------------------
>
> Perhaps it was the realclean that deleted all the .M3EXPORTS files.
> ../boot_cm3-core.sh LINUXLIBC6 gave this failure:
>
> -----------------------------------------------------------------------------------------------------------
>
> rodney at yellowstone:~/proj/m3/cm3-new/cm3/scripts$ ./boot-cm3-core.sh
> please specify a cross compilation target platform
> rodney at yellowstone:~/proj/m3/cm3-new/cm3/scripts$ ./boot-cm3-core.sh
> LINUXLIBC6
> /home/rodney/proj/m3/cm3-new/cm3/scripts/pkgmap.sh -c "cm3 -build
> -override -DROOT='/home/rodney/proj/m3/cm3-new/cm3'
> -DCM3_VERSION_TEXT='d5.7.0' -DCM3_VERSION_NUMBER='050700'
> -DCM3_LAST_CHANGED='2008-03-16' -DM3_BOOTSTRAP=TRUE -keep" m3core libm3
> sysutils m3middle m3objfile m3linker m3back m3front m3quake cm3
> m3scanner m3tools m3cgcat m3cggen m3bundle bitvector digraph
> parseparams realgeometry set slisp sortedtableextras table-list
> tempfiles
> === package /home/rodney/proj/m3/cm3-new/cm3/m3-libs/m3core ===
>  +++ cm3 -build -override -DROOT='/home/rodney/proj/m3/cm3-new/cm3'
> -DCM3_VERSION_TEXT='d5.7.0' -DCM3_VERSION_NUMBER='050700'
> -DCM3_LAST_CHANGED='2008-03-16' -DM3_BOOTSTRAP=TRUE -keep +++
> --- building in LINUXLIBC6 ---
>
> new source -> compiling RTHooks.i3
> /bin/sh:
> /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3cc/LINUXLIBC6-LINUXLIBC6/cm3cg: not
> found
>   m3_backend => 127
> m3cc (aka cm3cg) failed compiling: RTHooks.ic
>
> ---------------------------------------------------------------------------------------------------------
>
> Fortunately, an earlier ./upgrade.sh had built a cm3cg (though it had failed
> on all the Modula-3 code).  I got around the above by putting in a symbolic
> link from cm3/m3-sys/m3cc/LINUXLIBC6-LINUXLIBC6 to   
> cm3/m3-sys/m3cc/LINUXLIBC6,
> so it could find cm3cg.
>
> Then I went through the package-at-a-time order Tony had given in an old post
> to the m3devel list.  That had other problems.  Running cm3 in the package
> directory failed with the undefined ROOT problem, but do-pkg.sh would do the
> build.  But do-pkg.sh doesn't appear to do a ship, so I had to do that in
> each package with cm3 -ship.
>
> After building and shipping m3 middle and m3front, package cm3   
> itself wouldn't
> link, wanting libraries for m3back, m3objfile, m3quake, sysutils,
> m3linker, libm3,
> and m3core.  I skipped it and m3gc*, which don't exist now, and went
> through the
> rest of the order, which completed without failures building.  But that gave
> me a compiler that fails with a new problem I have never seen before:
>
> ---------------------------------------------------------------------------------------------------------
>
> rodney at yellowstone:~/proj/m3/cm3-new/cm3/scripts$
> ./install-cm3-compiler.sh upgrade
> cp_if: /usr/local/cm3/bin/cm3 and /usr/local/cm3/bin/cm3-d5.7.0 identical
> cp_if: /usr/local/cm3/bin/cm3cg and /usr/local/cm3/bin/cm3cg-d5.7.0 identical
>
>
> ***
> *** runtime error:
> ***    A compile-time type is missing.
> ***    file "../src/Utils.m3"
> ***
>
> Aborted (core dumped)
> cp /home/rodney/proj/m3/cm3-new/cm3/m3-sys/cm3/LINUXLIBC6/cm3
> /usr/local/cm3/bin/cm3-
> cp /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3cc/LINUXLIBC6/cm3cg
> /usr/local/cm3/bin/cm3cg-
> cp /usr/local/cm3/bin/cm3- /usr/local/cm3/bin/cm3
> cp /usr/local/cm3/bin/cm3cg- /usr/local/cm3/bin/cm3cg
>
> ------------------------------------------------------------------------------------------------------------
>
> Going back to an older cm3 makes that go away and makes the ROOT problem
> recur.  Working around that with cm3   
> -DROOT="/home/rodney/proj/m3/cm3-new/cm3"
> get the compiler to run, but it doesn't do anything, because it thinks
> nothing needs to be recompiled.  I could delete stuff, but I fear that would
> just start me around another iteration of the same thing.
>
> I don't believe I have ever gotten a bootstrap to work.  I think I   
> have always
> had to download a recent binary and go from there.
>
> Any help would be greatly appreciated.
>
> Rodney Bates



-- 
Olaf Wagner -- elego Software Solutions GmbH
                Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany
phone: +49 30 23 45 86 96  mobile: +49 177 2345 869  fax: +49 30 23 45 86 95
    http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin
Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194



From jay.krell at cornell.edu  Wed Nov 12 08:57:31 2008
From: jay.krell at cornell.edu (Jay)
Date: Wed, 12 Nov 2008 07:57:31 +0000
Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
In-Reply-To: <B9A95C1E-B873-40A9-A045-CD38E32CA5D0@cs.purdue.edu>
References: <COL101-W3DB0BA622C4D31A82B602E6180@phx.gbl>
	<C47F6B5A-F154-4485-B839-96185B59E3B6@cs.purdue.edu>
	<COL101-W2CFF8666D1486BD514159E61B0@phx.gbl> 
	<B9A95C1E-B873-40A9-A045-CD38E32CA5D0@cs.purdue.edu> 
Message-ID: <BAY116-W5161CD049D8CA0E7ABAE4AE6140@phx.gbl>


Target.Little_endian is used to drive some target-specificity in the IR.
   I guess, after all, that is why you have to specify it.
 
The IR for RTLinker.m3 is identical for MIPS64_OPENBSD and SPARC64_SOLARIS, once I set them both to unaligned_procedures, which they both need. (Tangent, I ran m3cgcat on the .ms file a few times instead of the .mc file; that makes it crash.)
 
I can't claim SPARC64_SOLARIS is any paragon of working and correct though, by far.
I should also be able to compare with SPARC64_OPENBSD, I forgot about that, and didn't have an m3cg.
I only ran SPARC64_SOLARIS far enough to discover it needed unaligned procedures.
I think I should fork linux-common and solaris-common off of the minimal 29k openbsd-common, rather than worry about all the stuff in solaris-2-x and linux-libc, over 100k each. Then SPARC64_SOLARIS could/would be in better shape probably.
 
MIPS64_OPENBSD cm3 gets further with unaligned_procedures, along with the parse.c fix -- it gets to a bus error in RunMainBody.
Again an alignment problem, though I think maybe memory memory trashing.
The fault occurs here:
 
    desc_offset := init_depth * ADRSIZE (InitDesc);
      desc := init_stack + desc_offset;      init_depth := desc.low_link; << 
  InitDesc = RECORD    module   : RT0.ModulePtr;    low_link : INTEGER;  END;
desc_offset appears to not be a multiple of 4.
 
I will look into that.
 
Long story short -- consider again the parse.c change?
I admit I still haven't debugged it (or rather, the code that consumes the data produced by it).
 
 - Jay



From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduCC: m3devel at elegosoft.comSubject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Sun, 9 Nov 2008 11:10:54 +0000

I agree. I am suspicious too. I will investigate more. Thanks.  - Jay



From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 9 Nov 2008 10:01:58 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
Aren't the integer values supposed to be written endian-independently in the IR?  I am suspicious about these differences.




On 9 Nov 2008, at 01:25, Jay wrote:

Thanks for the tip/direction. Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the same" but for two systematic diffs:  - AMD64_LINUX has a bunch of added calls to check_nil.   This is a "bug" in it, inoptimal but ok code, because "first readable address" is 0. It should probably be 4k. I should write a program that iterates starting at 0 to find the value.  (or better yet, cm3 should have a -configure option that does this? -- for configuration values for which there is a safe default; in general, imho, the targets should be auto configured via C, assuming there is a native development environment available).  - some integers are different (again, systematically)  AMD64_LINUX: init_int  656 13 Int.64 init_chars  664 "    typecode " init_int  680 2 Int.64 init_var  688 v.1 0 init_int  696 13 Int.64 init_chars  704 "    typeid   " init_int  720 2 Int.64 init_var  728 v.1 0 init_int  736 13 Int.64 init_chars  744 "RunMainBody: " init_int  760 2 Int.64 init_var  768 v.1 0 init_int  776 19 Int.64 init_chars  784 "RunMainBody: exec: " init_int  808 2 Int.64 init_var  816 v.1 0 init_int  824 17 Int.64 init_chars  832 "ExpandInitStack: "MIPS64_OPENBSD:  init_chars  664 "    typecode " init_int  680 8796093022208 Int.64 init_var  688 v.1 0 init_int  696 13 Int.64 init_chars  704 "    typeid   " init_int  720 8796093022208 Int.64 init_var  728 v.1 0 init_int  736 13 Int.64 init_chars  744 "RunMainBody: " init_int  760 8796093022208 Int.64 init_var  768 v.1 0 init_int  776 19 Int.64 init_chars  784 "RunMainBody: exec: " init_int  808 8796093022208 Int.64 init_var  816 v.1 0This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where little endian has 2. I don't know what these numbers represent.I should look at other systems -- preferably a working 64 bit big endian system, if there are any. Could be flags/type with text literals? (Btw, I always leave the TextLiteral.i3 change on my system, lowering the maximum literal length to around 4gig even on 64 bit targets, so 32 bit can bootstrap 64 bit...need to fix the compiler here to do target arithmetic instead of host arithmetic...) Thanks, - Jay

From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sat, 8 Nov 2008 22:55:54 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Have you checked to make sure that the pre-gcc IR makes sense?  Does it see these values as 32-bit or 64-bit?  The reason I ask is that there may be something wrong in your m3middle target specs that cause bad IR.  Once we know the IR looks right then we can try and fix the problem.



Antony Hosking | Associate Professor | Computer Science | Purdue University
305 N. University Street | West Lafayette | IN 47907 | USA
Office +1 765 494 6001 | Mobile +1 765 427 5484


On 8 Nov 2008, at 15:45, Jay wrote:

truncated as usual...along with losing a lot of whitespace..Let's see...Tony, how about switching these #if's?I explain some of why below.static voidm3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){ tree val;#if 0 if (o != 0 || TREE_TYPE (v) != dst_t) {   v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),    bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) {   v = m3_build1 (ADDR_EXPR, t_addr, v);   v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));   v = m3_build1 (INDIRECT_REF, dst_t,    m3_cast (build_pointer_type (dst_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) {   val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP ();}m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){#if 0 if (o != 0 || TREE_TYPE (v) != src_t) {   v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),    bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) {   v = m3_build1 (ADDR_EXPR, t_addr, v);   v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));   v = m3_build1 (INDIRECT_REF, src_t,    m3_cast (build_pointer_type (src_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) {   v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v);}I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).In an early version it seems that many loads/stores of global variables is wrong.The "first" code affected is RTLinker_ExpandModuleTable.PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS;  n_bytes: INTEGER; BEGIN   IF (modules = NIL) THEN     (* first time... *)     max_modules := InitialTableSize;     modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr));     IF (modules = NIL) THEN Cstdlib.abort (); END;   ELSE     n_bytes := max_modules * BYTESIZE (RT0.ModulePtr);     new_mods := Cstdlib.malloc (n_bytes + n_bytes);     IF (new_mods = NIL) THEN Cstdlib.abort (); END;     EVAL Cstring.memcpy (new_mods, modules, n_bytes);     Cstdlib.free (modules);     modules := new_mods;     INC (max_modules, max_modules);   END; END ExpandModuleTable;It should read back modules as NIL.What happens is it reads back modules as not NIL,and then max_modules should also be zero, but is "large", leading to malloc to fail,and abort called.The output of cm3cg -y is reasonable.It indicates offset loads.What is incorrect is that in many loads and stores, but not all, theoffsets are lost.(923) begin_procedure procedure RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable(924) set_source_line source line  208(925) load_nil(926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb(927) set_source_line source line  207(928) set_source_line source line  210(929) load_nil(930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xbHere is the incorrect code:.globl RTLinker__ExpandModuleTable.stabd 46,0,0.stabn 68,0,206,.LM108.LM108:.LFBB8:.set nomips16.ent RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable:.frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerld $2,0($2)      << WRONGbne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $3,MM_RTLinkerld $2,128($3)andi $2,$2,0x0ori $2,$2,0x1f4sd $2,128($3).stabn 68,0,213,.LM112.LM112:li $4,4000   # 0xfa0jal mallocsd $2,0($fp)ld $4,0($fp)dla $3,MM_RTLinkerld $2,136($3)andi $2,$2,0x0or $2,$4,$2sd $2,136($3)     << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerld $2,0($2)       << WRONG bne $2,$0,.L65jal abortb .L65.L61:.stabn 68,0,216,.LM114.LM114:dla $2,MM_RTLinkerld $2,0($2)        << WRONG dsll $2,$2,3sd $2,8($fp).stabn 68,0,217,.LM115.LM115:ld $3,8($fp)ld $2,8($fp)daddu $2,$3,$2move $4,$2jal mallocsd $2,0($fp)ld $2,0($fp)sd $2,16($fp).stabn 68,0,218,.LM116.LM116:ld $2,16($fp)bne $2,$0,.L64jal abort.L64:.stabn 68,0,219,.LM117.LM117:ld $4,16($fp)dla $2,MM_RTLinkerld $2,0($2)          << WRONGld $3,8($fp)move $5,$2move $6,$3jal memcpysd $2,0($fp).stabn 68,0,220,.LM118.LM118:dla $2,MM_RTLinkerld $2,0($2)           << WRONGmove $4,$2jal free.stabn 68,0,221,.LM119.LM119:ld $4,16($fp)dla $3,MM_RTLinkerld $2,136($3)         << OKandi $2,$2,0x0or $2,$4,$2sd $2,136($3)         << OK .stabn 68,0,222,.LM120.LM120:dla $2,MM_RTLinkerld $3,0($2)           << WRONGdla $2,MM_RTLinkerld $2,0($2)           << WRONGdaddu $4,$3,$2dla $3,MM_RTLinkerld $2,128($3)         << OKandi $2,$2,0x0or $2,$2,$4sd $2,128($3)         << OK.L65:.LBE9:.stabn 68,0,224,.LM121.LM121:move $sp,$fpld $31,56($sp)ld $fp,48($sp)ld $28,40($sp)daddiu $sp,$sp,64j $31.end RTLinker__ExpandModuleTableHere is corrected code:RTLinker__ExpandModuleTable:.frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerdaddiu $2,$2,136     << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2)bne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $2,MM_RTLinkerdaddiu $3,$2,128     << CORRECT li $2,500   # 0x1f4sd $2,0($3).stabn 68,0,213,.LM112.LM112:li $4,4000   # 0xfa0jal mallocsd $2,0($fp)dla $2,MM_RTLinkerdaddiu $3,$2,136      << CORRECT ld $2,0($fp)sd $2,0($3).stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerdaddiu $2,$2,136      << CORRECT ld $2,0($2)bne $2,$0,.L65jal abortb .L65.L61:I still dump core with this change, but I get much further.I have not tested this change at all otherwise -- i.e. on working platforms.I have not debugged through the gcc/m3cg code to see whythe offsets are only sometimes lost, or why other platformshave no problem. Is MIPS's "global pointer" unusual these days?I know SPARC has similar, and all the non-x86, non-AMD64 NT platformshad similar, but that's essentially nothing now (I'm not counting IA64).I know Linux has PLT (procedure linkage table) and GOT (global offset table).Same thing?I haven't looked much at other MIPS ports yet.http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8is what made it look about like it does -- in particularing using BIT_FIELD_REF.1.7:static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */    tree v;    int o;    tree src_t, dest_t;    m3_type src_T, dest_T;{ if (o == 0 && TREE_TYPE (v) == src_t) {   EXPR_PUSH (v); } else {   tree adr = m3_build1 (ADDR_EXPR, t_addr, v);   if (o != 0) {     adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT));   }   EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t,                         m3_cast (build_pointer_type (src_t), adr))); }#if 1 if (src_T != dest_T) {   EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) {   const char *name = "noname";   if (v != 0 && DECL_NAME(v) != 0) {     name = IDENTIFIER_POINTER(DECL_NAME(v));   }   fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n", name,           o, src_T, dest_T); }#else if (src_T != dest_T) {   if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) {     EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1));   } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) {     EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1));   } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) {     /* okay */   } else {     fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n",                 src_T, dest_T);   } }#endif}static voidm3_store (v, o, src_t, dest_t)  /* GCC32OK */    tree v;    int o;    tree src_t, dest_t;{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) {   rhs = EXPR_REF (-1); } else {   rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) {   lhs = v; } else {   tree f = make_node (FIELD_DECL);   TREE_TYPE (f) = dest_t;   DECL_ALIGN (f) = TYPE_ALIGN (dest_t);   DECL_SIZE (f) = TYPE_SIZE (dest_t);   DECL_MODE (f) = TYPE_MODE (dest_t);   DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT);   DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT);   DECL_FIELD_CONTEXT (f) = TREE_TYPE (v);   lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); }#if 1   expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);#else if (dest_t ==  NULL) {   expand_assignment (lhs, rhs, 0, 0); } else {   expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); }#endif EXPR_POP ();}1.8:static voidm3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */{ if (o == 0 && TREE_TYPE (v) == src_t) {   EXPR_PUSH (v); } else {   EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),    bitsize_int (o))); } if (src_T != dst_T) {   EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) {   const char *name = "noname";   if (v != 0 && DECL_NAME(v) != 0) {     name = IDENTIFIER_POINTER(DECL_NAME(v));   }   fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n", name,    o, src_T, dst_T); }}static voidm3_store (tree v, int o, tree src_t, tree dst_t)  /* GCC32OK */{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) {   rhs = EXPR_REF (-1); } else {   rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) {   lhs = v; } else {   lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),      bitsize_int (o)); } if (src_t != dst_t) {   rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP ();}- Jay<AMD64_LINUX_RTLinker.mc><MIPS64_OPENBSD_RTLinker.mc>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081112/afd2a674/attachment-0002.html>

From jay.krell at cornell.edu  Wed Nov 12 09:05:03 2008
From: jay.krell at cornell.edu (Jay)
Date: Wed, 12 Nov 2008 08:05:03 +0000
Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
In-Reply-To: <BAY116-W5161CD049D8CA0E7ABAE4AE6140@phx.gbl>
References: <COL101-W3DB0BA622C4D31A82B602E6180@phx.gbl>
	<C47F6B5A-F154-4485-B839-96185B59E3B6@cs.purdue.edu>
	<COL101-W2CFF8666D1486BD514159E61B0@phx.gbl> 
	<B9A95C1E-B873-40A9-A045-CD38E32CA5D0@cs.purdue.edu>  
	<BAY116-W5161CD049D8CA0E7ABAE4AE6140@phx.gbl>
Message-ID: <BAY121-W536D2BD8E7F1E500416F5FE6140@phx.gbl>


ps: it takes over 20 seconds for MIPS64_OPENBSD to crash in RunMainBody, which makes me suspect the platform is impractically slow.
I can only hope that optimization and/or not crashing makes it much faster.
The bootstrap build -- assemble a bunch, compile a little C, link, is also slow though.
System is 200MHz+ and 256meg RAM though, ought to suffice..
 
- Jay



From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduDate: Wed, 12 Nov 2008 07:57:31 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??

Target.Little_endian is used to drive some target-specificity in the IR.   I guess, after all, that is why you have to specify it. The IR for RTLinker.m3 is identical for MIPS64_OPENBSD and SPARC64_SOLARIS, once I set them both to unaligned_procedures, which they both need. (Tangent, I ran m3cgcat on the .ms file a few times instead of the .mc file; that makes it crash.) I can't claim SPARC64_SOLARIS is any paragon of working and correct though, by far.I should also be able to compare with SPARC64_OPENBSD, I forgot about that, and didn't have an m3cg.I only ran SPARC64_SOLARIS far enough to discover it needed unaligned procedures.I think I should fork linux-common and solaris-common off of the minimal 29k openbsd-common, rather than worry about all the stuff in solaris-2-x and linux-libc, over 100k each. Then SPARC64_SOLARIS could/would be in better shape probably. MIPS64_OPENBSD cm3 gets further with unaligned_procedures, along with the parse.c fix -- it gets to a bus error in RunMainBody.Again an alignment problem, though I think maybe memory memory trashing.The fault occurs here:     desc_offset := init_depth * ADRSIZE (InitDesc);      desc := init_stack + desc_offset;      init_depth := desc.low_link; <<   InitDesc = RECORD    module   : RT0.ModulePtr;    low_link : INTEGER;  END;desc_offset appears to not be a multiple of 4. I will look into that. Long story short -- consider again the parse.c change?I admit I still haven't debugged it (or rather, the code that consumes the data produced by it).  - Jay



From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduCC: m3devel at elegosoft.comSubject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Sun, 9 Nov 2008 11:10:54 +0000

I agree. I am suspicious too. I will investigate more. Thanks.  - Jay



From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 9 Nov 2008 10:01:58 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
Aren't the integer values supposed to be written endian-independently in the IR?  I am suspicious about these differences.




On 9 Nov 2008, at 01:25, Jay wrote:

Thanks for the tip/direction. Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the same" but for two systematic diffs:  - AMD64_LINUX has a bunch of added calls to check_nil.   This is a "bug" in it, inoptimal but ok code, because "first readable address" is 0. It should probably be 4k. I should write a program that iterates starting at 0 to find the value.  (or better yet, cm3 should have a -configure option that does this? -- for configuration values for which there is a safe default; in general, imho, the targets should be auto configured via C, assuming there is a native development environment available).  - some integers are different (again, systematically)  AMD64_LINUX: init_int  656 13 Int.64 init_chars  664 "    typecode " init_int  680 2 Int.64 init_var  688 v.1 0 init_int  696 13 Int.64 init_chars  704 "    typeid   " init_int  720 2 Int.64 init_var  728 v.1 0 init_int  736 13 Int.64 init_chars  744 "RunMainBody: " init_int  760 2 Int.64 init_var  768 v.1 0 init_int  776 19 Int.64 init_chars  784 "RunMainBody: exec: " init_int  808 2 Int.64 init_var  816 v.1 0 init_int  824 17 Int.64 init_chars  832 "ExpandInitStack: "MIPS64_OPENBSD:  init_chars  664 "    typecode " init_int  680 8796093022208 Int.64 init_var  688 v.1 0 init_int  696 13 Int.64 init_chars  704 "    typeid   " init_int  720 8796093022208 Int.64 init_var  728 v.1 0 init_int  736 13 Int.64 init_chars  744 "RunMainBody: " init_int  760 8796093022208 Int.64 init_var  768 v.1 0 init_int  776 19 Int.64 init_chars  784 "RunMainBody: exec: " init_int  808 8796093022208 Int.64 init_var  816 v.1 0This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where little endian has 2. I don't know what these numbers represent.I should look at other systems -- preferably a working 64 bit big endian system, if there are any. Could be flags/type with text literals? (Btw, I always leave the TextLiteral.i3 change on my system, lowering the maximum literal length to around 4gig even on 64 bit targets, so 32 bit can bootstrap 64 bit...need to fix the compiler here to do target arithmetic instead of host arithmetic...) Thanks, - Jay

From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sat, 8 Nov 2008 22:55:54 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Have you checked to make sure that the pre-gcc IR makes sense?  Does it see these values as 32-bit or 64-bit?  The reason I ask is that there may be something wrong in your m3middle target specs that cause bad IR.  Once we know the IR looks right then we can try and fix the problem.



Antony Hosking | Associate Professor | Computer Science | Purdue University
305 N. University Street | West Lafayette | IN 47907 | USA
Office +1 765 494 6001 | Mobile +1 765 427 5484


On 8 Nov 2008, at 15:45, Jay wrote:

truncated as usual...along with losing a lot of whitespace..Let's see...Tony, how about switching these #if's?I explain some of why below.static voidm3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){ tree val;#if 0 if (o != 0 || TREE_TYPE (v) != dst_t) {   v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),    bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) {   v = m3_build1 (ADDR_EXPR, t_addr, v);   v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));   v = m3_build1 (INDIRECT_REF, dst_t,    m3_cast (build_pointer_type (dst_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) {   val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP ();}m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){#if 0 if (o != 0 || TREE_TYPE (v) != src_t) {   v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),    bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) {   v = m3_build1 (ADDR_EXPR, t_addr, v);   v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));   v = m3_build1 (INDIRECT_REF, src_t,    m3_cast (build_pointer_type (src_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) {   v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v);}I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).In an early version it seems that many loads/stores of global variables is wrong.The "first" code affected is RTLinker_ExpandModuleTable.PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS;  n_bytes: INTEGER; BEGIN   IF (modules = NIL) THEN     (* first time... *)     max_modules := InitialTableSize;     modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr));     IF (modules = NIL) THEN Cstdlib.abort (); END;   ELSE     n_bytes := max_modules * BYTESIZE (RT0.ModulePtr);     new_mods := Cstdlib.malloc (n_bytes + n_bytes);     IF (new_mods = NIL) THEN Cstdlib.abort (); END;     EVAL Cstring.memcpy (new_mods, modules, n_bytes);     Cstdlib.free (modules);     modules := new_mods;     INC (max_modules, max_modules);   END; END ExpandModuleTable;It should read back modules as NIL.What happens is it reads back modules as not NIL,and then max_modules should also be zero, but is "large", leading to malloc to fail,and abort called.The output of cm3cg -y is reasonable.It indicates offset loads.What is incorrect is that in many loads and stores, but not all, theoffsets are lost.(923) begin_procedure procedure RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable(924) set_source_line source line  208(925) load_nil(926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb(927) set_source_line source line  207(928) set_source_line source line  210(929) load_nil(930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xbHere is the incorrect code:.globl RTLinker__ExpandModuleTable.stabd 46,0,0.stabn 68,0,206,.LM108.LM108:.LFBB8:.set nomips16.ent RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable:.frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerld $2,0($2)      << WRONGbne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $3,MM_RTLinkerld $2,128($3)andi $2,$2,0x0ori $2,$2,0x1f4sd $2,128($3).stabn 68,0,213,.LM112.LM112:li $4,4000   # 0xfa0jal mallocsd $2,0($fp)ld $4,0($fp)dla $3,MM_RTLinkerld $2,136($3)andi $2,$2,0x0or $2,$4,$2sd $2,136($3)     << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerld $2,0($2)       << WRONG bne $2,$0,.L65jal abortb .L65.L61:.stabn 68,0,216,.LM114.LM114:dla $2,MM_RTLinkerld $2,0($2)        << WRONG dsll $2,$2,3sd $2,8($fp).stabn 68,0,217,.LM115.LM115:ld $3,8($fp)ld $2,8($fp)daddu $2,$3,$2move $4,$2jal mallocsd $2,0($fp)ld $2,0($fp)sd $2,16($fp).stabn 68,0,218,.LM116.LM116:ld $2,16($fp)bne $2,$0,.L64jal abort.L64:.stabn 68,0,219,.LM117.LM117:ld $4,16($fp)dla $2,MM_RTLinkerld $2,0($2)          << WRONGld $3,8($fp)move $5,$2move $6,$3jal memcpysd $2,0($fp).stabn 68,0,220,.LM118.LM118:dla $2,MM_RTLinkerld $2,0($2)           << WRONGmove $4,$2jal free.stabn 68,0,221,.LM119.LM119:ld $4,16($fp)dla $3,MM_RTLinkerld $2,136($3)         << OKandi $2,$2,0x0or $2,$4,$2sd $2,136($3)         << OK .stabn 68,0,222,.LM120.LM120:dla $2,MM_RTLinkerld $3,0($2)           << WRONGdla $2,MM_RTLinkerld $2,0($2)           << WRONGdaddu $4,$3,$2dla $3,MM_RTLinkerld $2,128($3)         << OKandi $2,$2,0x0or $2,$2,$4sd $2,128($3)         << OK.L65:.LBE9:.stabn 68,0,224,.LM121.LM121:move $sp,$fpld $31,56($sp)ld $fp,48($sp)ld $28,40($sp)daddiu $sp,$sp,64j $31.end RTLinker__ExpandModuleTableHere is corrected code:RTLinker__ExpandModuleTable:.frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerdaddiu $2,$2,136     << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2)bne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $2,MM_RTLinkerdaddiu $3,$2,128     << CORRECT li $2,500   # 0x1f4sd $2,0($3).stabn 68,0,213,.LM112.LM112:li $4,4000   # 0xfa0jal mallocsd $2,0($fp)dla $2,MM_RTLinkerdaddiu $3,$2,136      << CORRECT ld $2,0($fp)sd $2,0($3).stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerdaddiu $2,$2,136      << CORRECT ld $2,0($2)bne $2,$0,.L65jal abortb .L65.L61:I still dump core with this change, but I get much further.I have not tested this change at all otherwise -- i.e. on working platforms.I have not debugged through the gcc/m3cg code to see whythe offsets are only sometimes lost, or why other platformshave no problem. Is MIPS's "global pointer" unusual these days?I know SPARC has similar, and all the non-x86, non-AMD64 NT platformshad similar, but that's essentially nothing now (I'm not counting IA64).I know Linux has PLT (procedure linkage table) and GOT (global offset table).Same thing?I haven't looked much at other MIPS ports yet.http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8is what made it look about like it does -- in particularing using BIT_FIELD_REF.1.7:static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */    tree v;    int o;    tree src_t, dest_t;    m3_type src_T, dest_T;{ if (o == 0 && TREE_TYPE (v) == src_t) {   EXPR_PUSH (v); } else {   tree adr = m3_build1 (ADDR_EXPR, t_addr, v);   if (o != 0) {     adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT));   }   EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t,                         m3_cast (build_pointer_type (src_t), adr))); }#if 1 if (src_T != dest_T) {   EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) {   const char *name = "noname";   if (v != 0 && DECL_NAME(v) != 0) {     name = IDENTIFIER_POINTER(DECL_NAME(v));   }   fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n", name,           o, src_T, dest_T); }#else if (src_T != dest_T) {   if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) {     EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1));   } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) {     EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1));   } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) {     /* okay */   } else {     fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n",                 src_T, dest_T);   } }#endif}static voidm3_store (v, o, src_t, dest_t)  /* GCC32OK */    tree v;    int o;    tree src_t, dest_t;{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) {   rhs = EXPR_REF (-1); } else {   rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) {   lhs = v; } else {   tree f = make_node (FIELD_DECL);   TREE_TYPE (f) = dest_t;   DECL_ALIGN (f) = TYPE_ALIGN (dest_t);   DECL_SIZE (f) = TYPE_SIZE (dest_t);   DECL_MODE (f) = TYPE_MODE (dest_t);   DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT);   DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT);   DECL_FIELD_CONTEXT (f) = TREE_TYPE (v);   lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); }#if 1   expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);#else if (dest_t ==  NULL) {   expand_assignment (lhs, rhs, 0, 0); } else {   expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); }#endif EXPR_POP ();}1.8:static voidm3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */{ if (o == 0 && TREE_TYPE (v) == src_t) {   EXPR_PUSH (v); } else {   EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),    bitsize_int (o))); } if (src_T != dst_T) {   EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) {   const char *name = "noname";   if (v != 0 && DECL_NAME(v) != 0) {     name = IDENTIFIER_POINTER(DECL_NAME(v));   }   fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n", name,    o, src_T, dst_T); }}static voidm3_store (tree v, int o, tree src_t, tree dst_t)  /* GCC32OK */{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) {   rhs = EXPR_REF (-1); } else {   rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) {   lhs = v; } else {   lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),      bitsize_int (o)); } if (src_t != dst_t) {   rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP ();}- Jay<AMD64_LINUX_RTLinker.mc><MIPS64_OPENBSD_RTLinker.mc>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081112/c2cc1a47/attachment-0002.html>

From rcoleburn at scires.com  Wed Nov 12 19:49:19 2008
From: rcoleburn at scires.com (Randy Coleburn)
Date: Wed, 12 Nov 2008 13:49:19 -0500
Subject: [M3devel] [M3commit] CVS Update: cm3
In-Reply-To: <20081112152142.02AEA10D5970@birch.elegosoft.com>
References: <20081112152142.02AEA10D5970@birch.elegosoft.com>
Message-ID: <491ADDBA.1E75.00D7.1@scires.com>

Rodney:
 
Is this change simply to make PM3 and CM3 more compatible wrt pickles, or does it alter current behavior for folks who only use CM3 ? (i.e., will it result in a change to the way CM3 pickles are represented that may affect current CM3 users?)
 
Regards,
Randy

>>> Rodney M. Bates <rodney at elego.de> 11/12/2008 4:21 PM >>>
CVSROOT:/usr/cvs
Changes by:rodney at birch.08/11/12 16:21:41

Modified files:
cm3/m3-libs/libm3/src/pickle/ver2/: Pickle2.m3 

Log message:
Pickles in both PM3 and CM3 have some hard-coded
fingerprints for common types, but the values
are not consistent about byte order.

Fix CM2 pickle-reading code to accept values of types
NULL, ROOT, UNTRACED ROOT, ADDRESS, AND REFANY,
written by PM3 pickle-writing code.  Probably,
the latter two can't even happen in a pickle, unless
somebody wrote a custom special for them.

This still won't handle a more complex type that depends
on these types, e.g., an object that contains a field of
one of them.  This latter is probably not feasable, because
it would require "decrypting" fingerprints, and the hash
algorithm for fingerprints is lossy.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081112/456771f5/attachment-0002.html>

From hosking at cs.purdue.edu  Wed Nov 12 21:46:53 2008
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Wed, 12 Nov 2008 20:46:53 +0000
Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
In-Reply-To: <BAY116-W5161CD049D8CA0E7ABAE4AE6140@phx.gbl>
References: <COL101-W3DB0BA622C4D31A82B602E6180@phx.gbl>
	<C47F6B5A-F154-4485-B839-96185B59E3B6@cs.purdue.edu>
	<COL101-W2CFF8666D1486BD514159E61B0@phx.gbl>
	<B9A95C1E-B873-40A9-A045-CD38E32CA5D0@cs.purdue.edu>
	<BAY116-W5161CD049D8CA0E7ABAE4AE6140@phx.gbl>
Message-ID: <80AE04BD-4819-4085-8EBE-5996E8E6DB93@cs.purdue.edu>

Jay, do you have access to an AMD64_DARWIN box?   I did get things  
working on that 64-bit platform.   Or is it an endianness issue you  
are dealing with.  In any case, I will not have a lot of bandwidth for  
doing much in the way of tackling this unless it becomes critical path  
for me soon.

Antony Hosking | Associate Professor | Computer Science | Purdue  
University
305 N. University Street | West Lafayette | IN 47907 | USA
Office +1 765 494 6001 | Mobile +1 765 427 5484



On 12 Nov 2008, at 07:57, Jay wrote:

> Target.Little_endian is used to drive some target-specificity in the  
> IR.
>    I guess, after all, that is why you have to specify it.
>
> The IR for RTLinker.m3 is identical for MIPS64_OPENBSD and  
> SPARC64_SOLARIS, once I set them both to unaligned_procedures, which  
> they both need. (Tangent, I ran m3cgcat on the .ms file a few times  
> instead of the .mc file; that makes it crash.)
>
> I can't claim SPARC64_SOLARIS is any paragon of working and correct  
> though, by far.
> I should also be able to compare with SPARC64_OPENBSD, I forgot  
> about that, and didn't have an m3cg.
> I only ran SPARC64_SOLARIS far enough to discover it needed  
> unaligned procedures.
> I think I should fork linux-common and solaris-common off of the  
> minimal 29k openbsd-common, rather than worry about all the stuff in  
> solaris-2-x and linux-libc, over 100k each. Then SPARC64_SOLARIS  
> could/would be in better shape probably.
>
> MIPS64_OPENBSD cm3 gets further with unaligned_procedures, along  
> with the parse.c fix -- it gets to a bus error in RunMainBody.
> Again an alignment problem, though I think maybe memory memory  
> trashing.
> The fault occurs here:
>
>     desc_offset := init_depth * ADRSIZE (InitDesc);
>
>       desc := init_stack + desc_offset;
>       init_depth := desc.low_link; <<
>
>   InitDesc = RECORD
>     module   : RT0.ModulePtr;
>     low_link : INTEGER;
>   END;
>
> desc_offset appears to not be a multiple of 4.
>
> I will look into that.
>
> Long story short -- consider again the parse.c change?
> I admit I still haven't debugged it (or rather, the code that  
> consumes the data produced by it).
>
>  - Jay
>
>
>
> From: jay.krell at cornell.edu
> To: hosking at cs.purdue.edu
> CC: m3devel at elegosoft.com
> Subject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
> Date: Sun, 9 Nov 2008 11:10:54 +0000
>
> I agree. I am suspicious too. I will investigate more. Thanks.
>
>  - Jay
>
>
>
>
> From: hosking at cs.purdue.edu
> To: jay.krell at cornell.edu
> Date: Sun, 9 Nov 2008 10:01:58 +0000
> CC: m3devel at elegosoft.com
> Subject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
>
>
>
> Aren't the integer values supposed to be written endian- 
> independently in the IR?  I am suspicious about these differences.
>
> On 9 Nov 2008, at 01:25, Jay wrote:
>
> Thanks for the tip/direction.
>
> Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the  
> same" but for two systematic diffs:
>
>  - AMD64_LINUX has a bunch of added calls to check_nil.
>    This is a "bug" in it, inoptimal but ok code, because "first  
> readable address" is 0. It should probably be 4k. I should write a  
> program that iterates starting at 0 to find the value.
>   (or better yet, cm3 should have a -configure option that does  
> this? -- for configuration values for which there is a safe default;  
> in general, imho, the targets should be auto configured via C,  
> assuming there is a native development environment available).
>
>  - some integers are different (again, systematically)
>
> AMD64_LINUX:
>  init_int  656 13 Int.64
>  init_chars  664 "    typecode "
>  init_int  680 2 Int.64
>  init_var  688 v.1 0
>  init_int  696 13 Int.64
>  init_chars  704 "    typeid   "
>  init_int  720 2 Int.64
>  init_var  728 v.1 0
>  init_int  736 13 Int.64
>  init_chars  744 "RunMainBody: "
>  init_int  760 2 Int.64
>  init_var  768 v.1 0
>  init_int  776 19 Int.64
>  init_chars  784 "RunMainBody: exec: "
>  init_int  808 2 Int.64
>  init_var  816 v.1 0
>  init_int  824 17 Int.64
>  init_chars  832 "ExpandInitStack: "
>
> MIPS64_OPENBSD:
>
>  init_chars  664 "    typecode "
>  init_int  680 8796093022208 Int.64
>  init_var  688 v.1 0
>  init_int  696 13 Int.64
>  init_chars  704 "    typeid   "
>  init_int  720 8796093022208 Int.64
>  init_var  728 v.1 0
>  init_int  736 13 Int.64
>  init_chars  744 "RunMainBody: "
>  init_int  760 8796093022208 Int.64
>  init_var  768 v.1 0
>  init_int  776 19 Int.64
>  init_chars  784 "RunMainBody: exec: "
>  init_int  808 8796093022208 Int.64
>  init_var  816 v.1 0
>
> This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where  
> little endian has 2.
>
> I don't know what these numbers represent.
> I should look at other systems -- preferably a working 64 bit big  
> endian system, if there are any.
>
> Could be flags/type with text literals?
>
> (Btw, I always leave the TextLiteral.i3 change on my system,  
> lowering the maximum literal length to around 4gig even on 64 bit  
> targets, so 32 bit can bootstrap 64 bit...need to fix the compiler  
> here to do target arithmetic instead of host arithmetic...)
>
> Thanks,
>  - Jay
>
>
> From: hosking at cs.purdue.edu
> To: jay.krell at cornell.edu
> Date: Sat, 8 Nov 2008 22:55:54 +0000
> CC: m3devel at elegosoft.com
> Subject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
>
> Have you checked to make sure that the pre-gcc IR makes sense?  Does  
> it see these values as 32-bit or 64-bit?  The reason I ask is that  
> there may be something wrong in your m3middle target specs that  
> cause bad IR.  Once we know the IR looks right then we can try and  
> fix the problem.
>
> Antony Hosking | Associate Professor | Computer Science | Purdue  
> University
> 305 N. University Street | West Lafayette | IN 47907 | USA
> Office +1 765 494 6001 | Mobile +1 765 427 5484
>
>
>
> On 8 Nov 2008, at 15:45, Jay wrote:
>
>
> truncated as usual...along with losing a lot of whitespace..
> Let's see...
>
>
> Tony, how about switching these #if's?
> I explain some of why below.
>
> static void
> m3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t,  
> m3_type dst_T)
> {
>  tree val;
> #if 0
>  if (o != 0 || TREE_TYPE (v) != dst_t) {
>    v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),
>     bitsize_int (o));
>  }
> #else
>  /* failsafe, but inefficient */
>  if (o != 0 || TREE_TYPE (v) != dst_t) {
>    v = m3_build1 (ADDR_EXPR, t_addr, v);
>    v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));
>    v = m3_build1 (INDIRECT_REF, dst_t,
>     m3_cast (build_pointer_type (dst_t), v));
>  }
> #endif
>  TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing  
> problems */
>  val = m3_cast (src_t, EXPR_REF (-1));
>  if (src_T != dst_T) {
>    val = m3_build1 (CONVERT_EXPR, dst_t, val);
>  }
>  add_stmt (build2 (MODIFY_EXPR, dst_t, v, val));
>  EXPR_POP ();
> }
> m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t,  
> m3_type dst_T)
> {
> #if 0
>  if (o != 0 || TREE_TYPE (v) != src_t) {
>    v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),
>     bitsize_int (o));
>  }
> #else
>  /* failsafe, but inefficient */
>  if (o != 0 || TREE_TYPE (v) != src_t) {
>    v = m3_build1 (ADDR_EXPR, t_addr, v);
>    v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));
>    v = m3_build1 (INDIRECT_REF, src_t,
>     m3_cast (build_pointer_type (src_t), v));
>  }
> #endif
>  TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing  
> problems */
>  if (src_T != dst_T) {
>    v = m3_build1 (CONVERT_EXPR, dst_t, v);
>  }
>  EXPR_PUSH (v);
> }
>
> I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).
> In an early version it seems that many loads/stores of global  
> variables is wrong.
> The "first" code affected is RTLinker_ExpandModuleTable.
>
> PROCEDURE ExpandModuleTable () =
>  CONST InitialTableSize = 500;
>  VAR new_mods: ADDRESS;  n_bytes: INTEGER;
>  BEGIN
>    IF (modules = NIL) THEN
>      (* first time... *)
>      max_modules := InitialTableSize;
>      modules := Cstdlib.malloc (InitialTableSize * BYTESIZE  
> (RT0.ModulePtr));
>      IF (modules = NIL) THEN Cstdlib.abort (); END;
>    ELSE
>      n_bytes := max_modules * BYTESIZE (RT0.ModulePtr);
>      new_mods := Cstdlib.malloc (n_bytes + n_bytes);
>      IF (new_mods = NIL) THEN Cstdlib.abort (); END;
>      EVAL Cstring.memcpy (new_mods, modules, n_bytes);
>      Cstdlib.free (modules);
>      modules := new_mods;
>      INC (max_modules, max_modules);
>    END;
>  END ExpandModuleTable;
>
> It should read back modules as NIL.
> What happens is it reads back modules as not NIL,
> and then max_modules should also be zero, but is "large", leading to  
> malloc to fail,
> and abort called.
>
> The output of cm3cg -y is reasonable.
> It indicates offset loads.
>
> What is incorrect is that in many loads and stores, but not all, the
> offsets are lost.
>
> (923) begin_procedure
>  procedure RTLinker__ExpandModuleTable
> RTLinker__ExpandModuleTable(924) set_source_line
>  source line  208
> (925) load_nil
> (926) store
>  store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb
> (927) set_source_line
>  source line  207
> (928) set_source_line
>  source line  210
> (929) load_nil
> (930) load
>  m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xb
>
> Here is the incorrect code:
>
> .globl RTLinker__ExpandModuleTable
> .stabd 46,0,0
> .stabn 68,0,206,.LM108
> .LM108:
> .LFBB8:
> .set nomips16
> .ent RTLinker__ExpandModuleTable
> RTLinker__ExpandModuleTable:
> .frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0
> .mask 0xd0000000,-8
> .fmask 0x00000000,0
> daddiu $sp,$sp,-64
> sd $31,56($sp)
> sd $fp,48($sp)
> sd $28,40($sp)
> move $fp,$sp
> lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
> daddu $28,$28,$25
> daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
> .LBB9:
> .stabn 68,0,208,.LM109
> .LM109:
> sd $0,16($fp)
> .stabn 68,0,210,.LM110
> .LM110:
> dla $2,MM_RTLinker
> ld $2,0($2)      << WRONG
> bne $2,$0,.L61
> .stabn 68,0,212,.LM111
> .LM111:
> dla $3,MM_RTLinker
> ld $2,128($3)
> andi $2,$2,0x0
> ori $2,$2,0x1f4
> sd $2,128($3)
> .stabn 68,0,213,.LM112
> .LM112:
> li $4,4000   # 0xfa0
> jal malloc
> sd $2,0($fp)
> ld $4,0($fp)
> dla $3,MM_RTLinker
> ld $2,136($3)
> andi $2,$2,0x0
> or $2,$4,$2
> sd $2,136($3)     << OK, offset seems low, but it agrees with  
> elsewhere
> .stabn 68,0,214,.LM113
> .LM113:
> dla $2,MM_RTLinker
> ld $2,0($2)       << WRONG
> bne $2,$0,.L65
> jal abort
> b .L65
> .L61:
> .stabn 68,0,216,.LM114
> .LM114:
> dla $2,MM_RTLinker
> ld $2,0($2)        << WRONG
> dsll $2,$2,3
> sd $2,8($fp)
> .stabn 68,0,217,.LM115
> .LM115:
> ld $3,8($fp)
> ld $2,8($fp)
> daddu $2,$3,$2
> move $4,$2
> jal malloc
> sd $2,0($fp)
> ld $2,0($fp)
> sd $2,16($fp)
> .stabn 68,0,218,.LM116
> .LM116:
> ld $2,16($fp)
> bne $2,$0,.L64
> jal abort
> .L64:
> .stabn 68,0,219,.LM117
> .LM117:
> ld $4,16($fp)
> dla $2,MM_RTLinker
> ld $2,0($2)          << WRONG
> ld $3,8($fp)
> move $5,$2
> move $6,$3
> jal memcpy
> sd $2,0($fp)
> .stabn 68,0,220,.LM118
> .LM118:
> dla $2,MM_RTLinker
> ld $2,0($2)           << WRONG
> move $4,$2
> jal free
> .stabn 68,0,221,.LM119
> .LM119:
> ld $4,16($fp)
> dla $3,MM_RTLinker
> ld $2,136($3)         << OK
> andi $2,$2,0x0
> or $2,$4,$2
> sd $2,136($3)         << OK
> .stabn 68,0,222,.LM120
> .LM120:
> dla $2,MM_RTLinker
> ld $3,0($2)           << WRONG
> dla $2,MM_RTLinker
> ld $2,0($2)           << WRONG
> daddu $4,$3,$2
> dla $3,MM_RTLinker
> ld $2,128($3)         << OK
> andi $2,$2,0x0
> or $2,$2,$4
> sd $2,128($3)         << OK
> .L65:
> .LBE9:
> .stabn 68,0,224,.LM121
> .LM121:
> move $sp,$fp
> ld $31,56($sp)
> ld $fp,48($sp)
> ld $28,40($sp)
> daddiu $sp,$sp,64
> j $31
> .end RTLinker__ExpandModuleTable
>
> Here is corrected code:
>
> RTLinker__ExpandModuleTable:
> .frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0
> .mask 0xd0000000,-8
> .fmask 0x00000000,0
> daddiu $sp,$sp,-64
> sd $31,56($sp)
> sd $fp,48($sp)
> sd $28,40($sp)
> move $fp,$sp
> lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
> daddu $28,$28,$25
> daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable)))
> .LBB9:
> .stabn 68,0,208,.LM109
> .LM109:
> sd $0,16($fp)
> .stabn 68,0,210,.LM110
> .LM110:
> dla $2,MM_RTLinker
> daddiu $2,$2,136     << CORRECT (offset seems low, but at least it  
> isn't zero)
> ld $2,0($2)
> bne $2,$0,.L61
> .stabn 68,0,212,.LM111
> .LM111:
> dla $2,MM_RTLinker
> daddiu $3,$2,128     << CORRECT
> li $2,500   # 0x1f4
> sd $2,0($3)
> .stabn 68,0,213,.LM112
> .LM112:
> li $4,4000   # 0xfa0
> jal malloc
> sd $2,0($fp)
> dla $2,MM_RTLinker
> daddiu $3,$2,136      << CORRECT
> ld $2,0($fp)
> sd $2,0($3)
> .stabn 68,0,214,.LM113
> .LM113:
> dla $2,MM_RTLinker
> daddiu $2,$2,136      << CORRECT
> ld $2,0($2)
> bne $2,$0,.L65
> jal abort
> b .L65
> .L61:
>
> I still dump core with this change, but I get much further.
> I have not tested this change at all otherwise -- i.e. on working  
> platforms.
>
> I have not debugged through the gcc/m3cg code to see why
> the offsets are only sometimes lost, or why other platforms
> have no problem. Is MIPS's "global pointer" unusual these days?
> I know SPARC has similar, and all the non-x86, non-AMD64 NT platforms
> had similar, but that's essentially nothing now (I'm not counting  
> IA64).
> I know Linux has PLT (procedure linkage table) and GOT (global  
> offset table).
> Same thing?
> I haven't looked much at other MIPS ports yet.
>
> http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8
>
> is what made it look about like it does -- in particularing using  
> BIT_FIELD_REF.
>
> 1.7:
>
> static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */
>     tree v;
>     int o;
>     tree src_t, dest_t;
>     m3_type src_T, dest_T;
> {
>  if (o == 0 && TREE_TYPE (v) == src_t) {
>    EXPR_PUSH (v);
>  } else {
>    tree adr = m3_build1 (ADDR_EXPR, t_addr, v);
>    if (o != 0) {
>      adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o /  
> BITS_PER_UNIT));
>    }
>    EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t,
>                          m3_cast (build_pointer_type (src_t), adr)));
>  }
> #if 1
>  if (src_T != dest_T) {
>    EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1));
>  }
>  if (debug_vars) {
>    const char *name = "noname";
>    if (v != 0 && DECL_NAME(v) != 0) {
>      name = IDENTIFIER_POINTER(DECL_NAME(v));
>    }
>    fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n",  
> name,
>            o, src_T, dest_T);
>  }
> #else
>  if (src_T != dest_T) {
>    if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) {
>      EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1));
>    } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) {
>      EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1));
>    } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) {
>      /* okay */
>    } else {
>      fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n",
>                  src_T, dest_T);
>    }
>  }
> #endif
> }
> static void
> m3_store (v, o, src_t, dest_t)  /* GCC32OK */
>     tree v;
>     int o;
>     tree src_t, dest_t;
> {
>  tree lhs, rhs;
>  if (TREE_TYPE (EXPR_REF (-1)) == src_t) {
>    rhs = EXPR_REF (-1);
>  } else {
>    rhs = m3_cast (src_t, EXPR_REF (-1));
>  }
>  if (o == 0 && TREE_TYPE (v) == dest_t) {
>    lhs = v;
>  } else {
>    tree f = make_node (FIELD_DECL);
>    TREE_TYPE (f) = dest_t;
>    DECL_ALIGN (f) = TYPE_ALIGN (dest_t);
>    DECL_SIZE (f) = TYPE_SIZE (dest_t);
>    DECL_MODE (f) = TYPE_MODE (dest_t);
>    DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT);
>    DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT);
>    DECL_FIELD_CONTEXT (f) = TREE_TYPE (v);
>    lhs = m3_build2 (COMPONENT_REF, dest_t, v, f);
>  }
> #if 1
>    expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0,  
> 0);
> #else
>  if (dest_t ==  NULL) {
>    expand_assignment (lhs, rhs, 0, 0);
>  } else {
>    expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0,  
> 0);
>  }
> #endif
>  EXPR_POP ();
> }
>
> 1.8:
> static void
> m3_load (tree v, int o,
>  tree src_t, m3_type src_T,
>  tree dst_t, m3_type dst_T) /* GCC32OK */
> {
>  if (o == 0 && TREE_TYPE (v) == src_t) {
>    EXPR_PUSH (v);
>  } else {
>    EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),
>     bitsize_int (o)));
>  }
>  if (src_T != dst_T) {
>    EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1));
>  }
>  if (option_vars_trace) {
>    const char *name = "noname";
>    if (v != 0 && DECL_NAME(v) != 0) {
>      name = IDENTIFIER_POINTER(DECL_NAME(v));
>    }
>    fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n",  
> name,
>     o, src_T, dst_T);
>  }
> }
> static void
> m3_store (tree v, int o, tree src_t, tree dst_t)  /* GCC32OK */
> {
>  tree lhs, rhs;
>  if (TREE_TYPE (EXPR_REF (-1)) == src_t) {
>    rhs = EXPR_REF (-1);
>  } else {
>    rhs = m3_cast (src_t, EXPR_REF (-1));
>  }
>  if (o == 0 && TREE_TYPE (v) == dst_t) {
>    lhs = v;
>  } else {
>    lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),
>       bitsize_int (o));
>  }
>  if (src_t != dst_t) {
>    rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs);
>  }
>  expand_assignment (lhs, rhs, 0, 0);
>  EXPR_POP ();
> }
>
> - Jay
>
> <AMD64_LINUX_RTLinker.mc><MIPS64_OPENBSD_RTLinker.mc>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081112/6950a8b4/attachment-0002.html>

From jay.krell at cornell.edu  Thu Nov 13 01:04:32 2008
From: jay.krell at cornell.edu (Jay)
Date: Thu, 13 Nov 2008 00:04:32 +0000
Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
In-Reply-To: <80AE04BD-4819-4085-8EBE-5996E8E6DB93@cs.purdue.edu>
References: <COL101-W3DB0BA622C4D31A82B602E6180@phx.gbl>
	<C47F6B5A-F154-4485-B839-96185B59E3B6@cs.purdue.edu>
	<COL101-W2CFF8666D1486BD514159E61B0@phx.gbl>
	<B9A95C1E-B873-40A9-A045-CD38E32CA5D0@cs.purdue.edu>
	<BAY116-W5161CD049D8CA0E7ABAE4AE6140@phx.gbl>
	<80AE04BD-4819-4085-8EBE-5996E8E6DB93@cs.purdue.edu>
Message-ID: <COL101-W60267BCFBF723F2491E3CEE6170@phx.gbl>


 > Jay, do you have access to an AMD64_DARWIN box?
 
No, sorry, I don't yet and not sure when I will.
Unless it is easy enough to run on non-Apple hardware.
I'll have to debug it (m3cg) then.
 
 - Jay



CC: m3devel at elegosoft.comFrom: hosking at cs.purdue.eduTo: jay.krell at cornell.eduSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Wed, 12 Nov 2008 20:46:53 +0000Jay, do you have access to an AMD64_DARWIN box?   I did get things working on that 64-bit platform.   Or is it an endianness issue you are dealing with.  In any case, I will not have a lot of bandwidth for doing much in the way of tackling this unless it becomes critical path for me soon.




Antony Hosking | Associate Professor | Computer Science | Purdue University
305 N. University Street | West Lafayette | IN 47907 | USA
Office +1 765 494 6001 | Mobile +1 765 427 5484


On 12 Nov 2008, at 07:57, Jay wrote:

Target.Little_endian is used to drive some target-specificity in the IR.   I guess, after all, that is why you have to specify it. The IR for RTLinker.m3 is identical for MIPS64_OPENBSD and SPARC64_SOLARIS, once I set them both to unaligned_procedures, which they both need. (Tangent, I ran m3cgcat on the .ms file a few times instead of the .mc file; that makes it crash.) I can't claim SPARC64_SOLARIS is any paragon of working and correct though, by far.I should also be able to compare with SPARC64_OPENBSD, I forgot about that, and didn't have an m3cg.I only ran SPARC64_SOLARIS far enough to discover it needed unaligned procedures.I think I should fork linux-common and solaris-common off of the minimal 29k openbsd-common, rather than worry about all the stuff in solaris-2-x and linux-libc, over 100k each. Then SPARC64_SOLARIS could/would be in better shape probably. MIPS64_OPENBSD cm3 gets further with unaligned_procedures, along with the parse.c fix -- it gets to a bus error in RunMainBody.Again an alignment problem, though I think maybe memory memory trashing.The fault occurs here:     desc_offset := init_depth * ADRSIZE (InitDesc);      desc := init_stack + desc_offset;      init_depth := desc.low_link; <<   InitDesc = RECORD    module   : RT0.ModulePtr;    low_link : INTEGER;  END;desc_offset appears to not be a multiple of 4. I will look into that. Long story short -- consider again the parse.c change?I admit I still haven't debugged it (or rather, the code that consumes the data produced by it).  - Jay

From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduCC: m3devel at elegosoft.comSubject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Sun, 9 Nov 2008 11:10:54 +0000I agree. I am suspicious too. I will investigate more. Thanks.  - Jay

From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 9 Nov 2008 10:01:58 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??
Aren't the integer values supposed to be written endian-independently in the IR?  I am suspicious about these differences.




On 9 Nov 2008, at 01:25, Jay wrote:

Thanks for the tip/direction. Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the same" but for two systematic diffs:  - AMD64_LINUX has a bunch of added calls to check_nil.   This is a "bug" in it, inoptimal but ok code, because "first readable address" is 0. It should probably be 4k. I should write a program that iterates starting at 0 to find the value.  (or better yet, cm3 should have a -configure option that does this? -- for configuration values for which there is a safe default; in general, imho, the targets should be auto configured via C, assuming there is a native development environment available).  - some integers are different (again, systematically)  AMD64_LINUX: init_int  656 13 Int.64 init_chars  664 "    typecode " init_int  680 2 Int.64 init_var  688 v.1 0 init_int  696 13 Int.64 init_chars  704 "    typeid   " init_int  720 2 Int.64 init_var  728 v.1 0 init_int  736 13 Int.64 init_chars  744 "RunMainBody: " init_int  760 2 Int.64 init_var  768 v.1 0 init_int  776 19 Int.64 init_chars  784 "RunMainBody: exec: " init_int  808 2 Int.64 init_var  816 v.1 0 init_int  824 17 Int.64 init_chars  832 "ExpandInitStack: "MIPS64_OPENBSD:  init_chars  664 "    typecode " init_int  680 8796093022208 Int.64 init_var  688 v.1 0 init_int  696 13 Int.64 init_chars  704 "    typeid   " init_int  720 8796093022208 Int.64 init_var  728 v.1 0 init_int  736 13 Int.64 init_chars  744 "RunMainBody: " init_int  760 8796093022208 Int.64 init_var  768 v.1 0 init_int  776 19 Int.64 init_chars  784 "RunMainBody: exec: " init_int  808 8796093022208 Int.64 init_var  816 v.1 0This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where little endian has 2. I don't know what these numbers represent.I should look at other systems -- preferably a working 64 bit big endian system, if there are any. Could be flags/type with text literals? (Btw, I always leave the TextLiteral.i3 change on my system, lowering the maximum literal length to around 4gig even on 64 bit targets, so 32 bit can bootstrap 64 bit...need to fix the compiler here to do target arithmetic instead of host arithmetic...) Thanks, - Jay

From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sat, 8 Nov 2008 22:55:54 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Have you checked to make sure that the pre-gcc IR makes sense?  Does it see these values as 32-bit or 64-bit?  The reason I ask is that there may be something wrong in your m3middle target specs that cause bad IR.  Once we know the IR looks right then we can try and fix the problem.



Antony Hosking | Associate Professor | Computer Science | Purdue University
305 N. University Street | West Lafayette | IN 47907 | USA
Office +1 765 494 6001 | Mobile +1 765 427 5484


On 8 Nov 2008, at 15:45, Jay wrote:

truncated as usual...along with losing a lot of whitespace..Let's see...Tony, how about switching these #if's?I explain some of why below.static voidm3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){ tree val;#if 0 if (o != 0 || TREE_TYPE (v) != dst_t) {   v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),    bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) {   v = m3_build1 (ADDR_EXPR, t_addr, v);   v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));   v = m3_build1 (INDIRECT_REF, dst_t,    m3_cast (build_pointer_type (dst_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) {   val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP ();}m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){#if 0 if (o != 0 || TREE_TYPE (v) != src_t) {   v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),    bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) {   v = m3_build1 (ADDR_EXPR, t_addr, v);   v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT));   v = m3_build1 (INDIRECT_REF, src_t,    m3_cast (build_pointer_type (src_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) {   v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v);}I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).In an early version it seems that many loads/stores of global variables is wrong.The "first" code affected is RTLinker_ExpandModuleTable.PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS;  n_bytes: INTEGER; BEGIN   IF (modules = NIL) THEN     (* first time... *)     max_modules := InitialTableSize;     modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr));     IF (modules = NIL) THEN Cstdlib.abort (); END;   ELSE     n_bytes := max_modules * BYTESIZE (RT0.ModulePtr);     new_mods := Cstdlib.malloc (n_bytes + n_bytes);     IF (new_mods = NIL) THEN Cstdlib.abort (); END;     EVAL Cstring.memcpy (new_mods, modules, n_bytes);     Cstdlib.free (modules);     modules := new_mods;     INC (max_modules, max_modules);   END; END ExpandModuleTable;It should read back modules as NIL.What happens is it reads back modules as not NIL,and then max_modules should also be zero, but is "large", leading to malloc to fail,and abort called.The output of cm3cg -y is reasonable.It indicates offset loads.What is incorrect is that in many loads and stores, but not all, theoffsets are lost.(923) begin_procedure procedure RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable(924) set_source_line source line  208(925) load_nil(926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb(927) set_source_line source line  207(928) set_source_line source line  210(929) load_nil(930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xbHere is the incorrect code:.globl RTLinker__ExpandModuleTable.stabd 46,0,0.stabn 68,0,206,.LM108.LM108:.LFBB8:.set nomips16.ent RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable:.frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerld $2,0($2)      << WRONGbne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $3,MM_RTLinkerld $2,128($3)andi $2,$2,0x0ori $2,$2,0x1f4sd $2,128($3).stabn 68,0,213,.LM112.LM112:li $4,4000   # 0xfa0jal mallocsd $2,0($fp)ld $4,0($fp)dla $3,MM_RTLinkerld $2,136($3)andi $2,$2,0x0or $2,$4,$2sd $2,136($3)     << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerld $2,0($2)       << WRONG bne $2,$0,.L65jal abortb .L65.L61:.stabn 68,0,216,.LM114.LM114:dla $2,MM_RTLinkerld $2,0($2)        << WRONG dsll $2,$2,3sd $2,8($fp).stabn 68,0,217,.LM115.LM115:ld $3,8($fp)ld $2,8($fp)daddu $2,$3,$2move $4,$2jal mallocsd $2,0($fp)ld $2,0($fp)sd $2,16($fp).stabn 68,0,218,.LM116.LM116:ld $2,16($fp)bne $2,$0,.L64jal abort.L64:.stabn 68,0,219,.LM117.LM117:ld $4,16($fp)dla $2,MM_RTLinkerld $2,0($2)          << WRONGld $3,8($fp)move $5,$2move $6,$3jal memcpysd $2,0($fp).stabn 68,0,220,.LM118.LM118:dla $2,MM_RTLinkerld $2,0($2)           << WRONGmove $4,$2jal free.stabn 68,0,221,.LM119.LM119:ld $4,16($fp)dla $3,MM_RTLinkerld $2,136($3)         << OKandi $2,$2,0x0or $2,$4,$2sd $2,136($3)         << OK .stabn 68,0,222,.LM120.LM120:dla $2,MM_RTLinkerld $3,0($2)           << WRONGdla $2,MM_RTLinkerld $2,0($2)           << WRONGdaddu $4,$3,$2dla $3,MM_RTLinkerld $2,128($3)         << OKandi $2,$2,0x0or $2,$2,$4sd $2,128($3)         << OK.L65:.LBE9:.stabn 68,0,224,.LM121.LM121:move $sp,$fpld $31,56($sp)ld $fp,48($sp)ld $28,40($sp)daddiu $sp,$sp,64j $31.end RTLinker__ExpandModuleTableHere is corrected code:RTLinker__ExpandModuleTable:.frame $fp,64,$31  # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerdaddiu $2,$2,136     << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2)bne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $2,MM_RTLinkerdaddiu $3,$2,128     << CORRECT li $2,500   # 0x1f4sd $2,0($3).stabn 68,0,213,.LM112.LM112:li $4,4000   # 0xfa0jal mallocsd $2,0($fp)dla $2,MM_RTLinkerdaddiu $3,$2,136      << CORRECT ld $2,0($fp)sd $2,0($3).stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerdaddiu $2,$2,136      << CORRECT ld $2,0($2)bne $2,$0,.L65jal abortb .L65.L61:I still dump core with this change, but I get much further.I have not tested this change at all otherwise -- i.e. on working platforms.I have not debugged through the gcc/m3cg code to see whythe offsets are only sometimes lost, or why other platformshave no problem. Is MIPS's "global pointer" unusual these days?I know SPARC has similar, and all the non-x86, non-AMD64 NT platformshad similar, but that's essentially nothing now (I'm not counting IA64).I know Linux has PLT (procedure linkage table) and GOT (global offset table).Same thing?I haven't looked much at other MIPS ports yet.http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8is what made it look about like it does -- in particularing using BIT_FIELD_REF.1.7:static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */    tree v;    int o;    tree src_t, dest_t;    m3_type src_T, dest_T;{ if (o == 0 && TREE_TYPE (v) == src_t) {   EXPR_PUSH (v); } else {   tree adr = m3_build1 (ADDR_EXPR, t_addr, v);   if (o != 0) {     adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT));   }   EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t,                         m3_cast (build_pointer_type (src_t), adr))); }#if 1 if (src_T != dest_T) {   EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) {   const char *name = "noname";   if (v != 0 && DECL_NAME(v) != 0) {     name = IDENTIFIER_POINTER(DECL_NAME(v));   }   fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n", name,           o, src_T, dest_T); }#else if (src_T != dest_T) {   if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) {     EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1));   } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) {     EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1));   } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) {     /* okay */   } else {     fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n",                 src_T, dest_T);   } }#endif}static voidm3_store (v, o, src_t, dest_t)  /* GCC32OK */    tree v;    int o;    tree src_t, dest_t;{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) {   rhs = EXPR_REF (-1); } else {   rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) {   lhs = v; } else {   tree f = make_node (FIELD_DECL);   TREE_TYPE (f) = dest_t;   DECL_ALIGN (f) = TYPE_ALIGN (dest_t);   DECL_SIZE (f) = TYPE_SIZE (dest_t);   DECL_MODE (f) = TYPE_MODE (dest_t);   DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT);   DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT);   DECL_FIELD_CONTEXT (f) = TREE_TYPE (v);   lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); }#if 1   expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);#else if (dest_t ==  NULL) {   expand_assignment (lhs, rhs, 0, 0); } else {   expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); }#endif EXPR_POP ();}1.8:static voidm3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */{ if (o == 0 && TREE_TYPE (v) == src_t) {   EXPR_PUSH (v); } else {   EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t),    bitsize_int (o))); } if (src_T != dst_T) {   EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) {   const char *name = "noname";   if (v != 0 && DECL_NAME(v) != 0) {     name = IDENTIFIER_POINTER(DECL_NAME(v));   }   fprintf(stderr, "  m3_load (%s): offset %d, convert %d -> %d\n", name,    o, src_T, dst_T); }}static voidm3_store (tree v, int o, tree src_t, tree dst_t)  /* GCC32OK */{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) {   rhs = EXPR_REF (-1); } else {   rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) {   lhs = v; } else {   lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t),      bitsize_int (o)); } if (src_t != dst_t) {   rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP ();}- Jay<AMD64_LINUX_RTLinker.mc><MIPS64_OPENBSD_RTLinker.mc>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081113/2f8b22da/attachment-0002.html>

From rodney.bates at wichita.edu  Thu Nov 13 16:45:34 2008
From: rodney.bates at wichita.edu (Rodney M. Bates)
Date: Thu, 13 Nov 2008 09:45:34 -0600
Subject: [M3devel] Still can't bootstrap cm3
In-Reply-To: <20081112081855.alr1p7k4ggw0k4k0@mail.elegosoft.com>
References: <4919ED6D.6000700@wichita.edu>
	<20081112081855.alr1p7k4ggw0k4k0@mail.elegosoft.com>
Message-ID: <491C4B9E.4010307@wichita.edu>

Olaf Wagner wrote:
> 
> it's not easy to understand what has gone wrong at this point.
> Usually, if you have a working system, cm3/scripts/upgrade.sh
> should be able to bootstrap to a new set of sources. This is tested
> every day in the tinderbox release tests, too. As the last builds seem
> to have succeeded, I don't think there is a real problem in our
> code base currently. You may want to have a look at test_build_current
> in cm3/scripts/regression/defs.sh to see what's done in the regression
> builds.
> 
> I'd suggest you either start again from the last release or
> one of the latest snapshots available from the build machine.

Yes, that's what I always end up doing.  But it would be nice to be
able someday to get my local compiler to rebuild itself :-(.

But even that is not working for me.  Reinstalling a snapshot from
2008-6-6 and a fresh cvs checkout, upgrade.sh gives the missing
.M3EXPORTS failure, and do-cm3-core.sh (after realclean) gets bad
version stamps on Compiler.Platform and Compiler.ThisPlatform,
while compiling SocketPosix.m3.

Going to the most recent (at the time) snapshot of 2008-11-12,
do-cm3-core.sh will run without failures, but leaves a compiler
that always gets the undefined ROOT failure, even trying to compile
a trivial program.

> I assume you are not building a cross compiler, but just trying
> to perform an upgrade.

That's right.  The only reason I messed with the cross scripts was that
I hoped they might take care of the necessary bootstrapping in a way
the would work.

> 
> Daily snapshots are available from
> http://www.opencm3.net/snaps/snapshot-index.html
> 
> I hope this helps,
> 
> Olaf
> 
>


From jay.krell at cornell.edu  Thu Nov 13 17:37:47 2008
From: jay.krell at cornell.edu (Jay)
Date: Thu, 13 Nov 2008 16:37:47 +0000
Subject: [M3devel] Still can't bootstrap cm3
In-Reply-To: <491C4B9E.4010307@wichita.edu>
References: <4919ED6D.6000700@wichita.edu>
	<20081112081855.alr1p7k4ggw0k4k0@mail.elegosoft.com> 
	<491C4B9E.4010307@wichita.edu>
Message-ID: <BAY118-W1318AE445EC7D9ACF911E8E6170@phx.gbl>


Randy, ROOT not being defined is a function of your config files.
Look at mine. Mine are willing to ferry it from an environment variable.
The various scripts figure it out and set it on the command line.
 e.g. sysinfo.sh
 
Granted..I don't think ROOT is supposed to be required.
ROOT is the root of the source tree, as you correctly?successfully worked around.
You are supposed to be able to use the compiler without a source tree.
But you can't, um, build the source tree without a source tree.
 
I am very tempted to have cm3.exe set ROOT itself,
but it is a bit inappropriate actually.
 
PERHAPS if it was renamed CM3_ROOT...
 
Olaf isn't saying "just use a current release", he is
saying "start with a current release" and then try to build locally.
 
You cannot necessarily bootstrap from arbitrarily old releases.
But you should definitely be able to bootstrap from current snapshots.
 
It does work.
 
Whenever you get into a "bad state" -- after any failed run,
I encourage you to blow away the entire cm3 install and "start over".
 
Several "partial builds" will leave you with a "mixed state".
 
Some of the scripts do or used to "backup" the pkg store,
but I don't think it was worth the effort...the effort I put
in to clone the .sh in .cmd. I think I punted doing that in the *.py,
and I don't miss it.
 
I also recommend trying the *.py files.
 
> that always gets the undefined ROOT failure, even trying to compile> a trivial program.
Hm. Could be your config file wants it?
Could be my config files wants it?
I guess the usual refrain -- send us your config file(s).
I can dig more later..maybe days..busy with other stuff..
 
 
 - Jay> Date: Thu, 13 Nov 2008 09:45:34 -0600> From: rodney> To: m3devel> Subject: Re: [M3devel] Still can't bootstrap cm3> > Olaf wrote:> > > > it's not easy to understand what has gone wrong at this point.> > Usually, if you have a working system, cm3/scripts/upgrade.sh> > should be able to bootstrap to a new set of sources. This is tested> > every day in the tinderbox release tests, too. As the last builds seem> > to have succeeded, I don't think there is a real problem in our> > code base currently. You may want to have a look at test_build_current> > in cm3/scripts/regression/defs.sh to see what's done in the regression> > builds.> > > > I'd suggest you either start again from the last release or> > one of the latest snapshots available from the build machine.> > Yes, that's what I always end up doing. But it would be nice to be> able someday to get my local compiler to rebuild itself :-(.> > But even that is not working for me. Reinstalling a snapshot from> 2008-6-6 and a fresh cvs checkout, upgrade.sh gives the missing> .M3EXPORTS failure, and do-cm3-core.sh (after realclean) gets bad> version stamps on Compiler.Platform and Compiler.ThisPlatform,> while compiling SocketPosix.m3.> > Going to the most recent (at the time) snapshot of 2008-11-12,> do-cm3-core.sh will run without failures, but leaves a compiler> that always gets the undefined ROOT failure, even trying to compile> a trivial program.> > > I assume you are not building a cross compiler, but just trying> > to perform an upgrade.> > That's right. The only reason I messed with the cross scripts was that> I hoped they might take care of the necessary bootstrapping in a way> the would work.> > > > > Daily snapshots are available from> > http://www.opencm3.net/snaps/snapshot-index.html> > > > I hope this helps,> > > > Olaf> > > >
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081113/3998c75f/attachment-0002.html>

From jay.krell at cornell.edu  Thu Nov 13 17:54:55 2008
From: jay.krell at cornell.edu (Jay)
Date: Thu, 13 Nov 2008 16:54:55 +0000
Subject: [M3devel] Still can't bootstrap cm3
In-Reply-To: <BAY118-W1318AE445EC7D9ACF911E8E6170@phx.gbl>
References: <4919ED6D.6000700@wichita.edu>
	<20081112081855.alr1p7k4ggw0k4k0@mail.elegosoft.com> 
	<491C4B9E.4010307@wichita.edu> 
	<BAY118-W1318AE445EC7D9ACF911E8E6170@phx.gbl>
Message-ID: <BAY117-W27AE620521647E466EC8D2E6170@phx.gbl>


Also, until you find something that works:
 export OMIT_GCC=yes
 
Otherwise you are going to waste a lot more time experimenting.
 
 - Jay



From: jay.krell at cornell.eduTo: rodney.bates at wichita.edu; m3devel at elegosoft.comDate: Thu, 13 Nov 2008 16:37:47 +0000Subject: Re: [M3devel] Still can't bootstrap cm3

Randy, ROOT not being defined is a function of your config files.Look at mine. Mine are willing to ferry it from an environment variable.The various scripts figure it out and set it on the command line. e.g. sysinfo.sh Granted..I don't think ROOT is supposed to be required.ROOT is the root of the source tree, as you correctly?successfully worked around.You are supposed to be able to use the compiler without a source tree.But you can't, um, build the source tree without a source tree. I am very tempted to have cm3.exe set ROOT itself,but it is a bit inappropriate actually. PERHAPS if it was renamed CM3_ROOT... Olaf isn't saying "just use a current release", he issaying "start with a current release" and then try to build locally. You cannot necessarily bootstrap from arbitrarily old releases.But you should definitely be able to bootstrap from current snapshots. It does work. Whenever you get into a "bad state" -- after any failed run,I encourage you to blow away the entire cm3 install and "start over". Several "partial builds" will leave you with a "mixed state". Some of the scripts do or used to "backup" the pkg store,but I don't think it was worth the effort...the effort I putin to clone the .sh in .cmd. I think I punted doing that in the *.py,and I don't miss it. I also recommend trying the *.py files. > that always gets the undefined ROOT failure, even trying to compile> a trivial program.Hm. Could be your config file wants it?Could be my config files wants it?I guess the usual refrain -- send us your config file(s).I can dig more later..maybe days..busy with other stuff..   - Jay> Date: Thu, 13 Nov 2008 09:45:34 -0600> From: rodney> To: m3devel> Subject: Re: [M3devel] Still can't bootstrap cm3> > Olaf wrote:> > > > it's not easy to understand what has gone wrong at this point.> > Usually, if you have a working system, cm3/scripts/upgrade.sh> > should be able to bootstrap to a new set of sources. This is tested> > every day in the tinderbox release tests, too. As the last builds seem> > to have succeeded, I don't think there is a real problem in our> > code base currently. You may want to have a look at test_build_current> > in cm3/scripts/regression/defs.sh to see what's done in the regression> > builds.> > > > I'd suggest you either start again from the last release or> > one of the latest snapshots available from the build machine.> > Yes, that's what I always end up doing. But it would be nice to be> able someday to get my local compiler to rebuild itself :-(.> > But even that is not working for me. Reinstalling a snapshot from> 2008-6-6 and a fresh cvs checkout, upgrade.sh gives the missing> .M3EXPORTS failure, and do-cm3-core.sh (after realclean) gets bad> version stamps on Compiler.Platform and Compiler.ThisPlatform,> while compiling SocketPosix.m3.> > Going to the most recent (at the time) snapshot of 2008-11-12,> do-cm3-core.sh will run without failures, but leaves a compiler> that always gets the undefined ROOT failure, even trying to compile> a trivial program.> > > I assume you are not building a cross compiler, but just trying> > to perform an upgrade.> > That's right. The only reason I messed with the cross scripts was that> I hoped they might take care of the necessary bootstrapping in a way> the would work.> > > > > Daily snapshots are available from> > http://www.opencm3.net/snaps/snapshot-index.html> > > > I hope this helps,> > > > Olaf> > > >
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081113/00c9851a/attachment-0002.html>

From hosking at cs.purdue.edu  Thu Nov 13 18:06:22 2008
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Thu, 13 Nov 2008 11:06:22 -0600
Subject: [M3devel] Still can't bootstrap cm3
In-Reply-To: <491C4B9E.4010307@wichita.edu>
References: <4919ED6D.6000700@wichita.edu>
	<20081112081855.alr1p7k4ggw0k4k0@mail.elegosoft.com>
	<491C4B9E.4010307@wichita.edu>
Message-ID: <77D02692-31FE-4326-8E0E-8758C323CDE9@cs.purdue.edu>

This is odd.  I've been rebuilding my local compiler by hand several  
times in the last few weeks.  I will try the scripts and follow up.

Antony Hosking | Associate Professor | Computer Science | Purdue  
University
305 N. University Street | West Lafayette | IN 47907 | USA
Office +1 765 494 6001 | Mobile +1 765 427 5484



On 13 Nov 2008, at 09:45, Rodney M. Bates wrote:

> Olaf Wagner wrote:
>> it's not easy to understand what has gone wrong at this point.
>> Usually, if you have a working system, cm3/scripts/upgrade.sh
>> should be able to bootstrap to a new set of sources. This is tested
>> every day in the tinderbox release tests, too. As the last builds  
>> seem
>> to have succeeded, I don't think there is a real problem in our
>> code base currently. You may want to have a look at  
>> test_build_current
>> in cm3/scripts/regression/defs.sh to see what's done in the  
>> regression
>> builds.
>> I'd suggest you either start again from the last release or
>> one of the latest snapshots available from the build machine.
>
> Yes, that's what I always end up doing.  But it would be nice to be
> able someday to get my local compiler to rebuild itself :-(.
>
> But even that is not working for me.  Reinstalling a snapshot from
> 2008-6-6 and a fresh cvs checkout, upgrade.sh gives the missing
> .M3EXPORTS failure, and do-cm3-core.sh (after realclean) gets bad
> version stamps on Compiler.Platform and Compiler.ThisPlatform,
> while compiling SocketPosix.m3.
>
> Going to the most recent (at the time) snapshot of 2008-11-12,
> do-cm3-core.sh will run without failures, but leaves a compiler
> that always gets the undefined ROOT failure, even trying to compile
> a trivial program.
>
>> I assume you are not building a cross compiler, but just trying
>> to perform an upgrade.
>
> That's right.  The only reason I messed with the cross scripts was  
> that
> I hoped they might take care of the necessary bootstrapping in a way
> the would work.
>
>> Daily snapshots are available from
>> http://www.opencm3.net/snaps/snapshot-index.html
>> I hope this helps,
>> Olaf
>>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081113/d9f9e670/attachment-0002.html>

From rodney.bates at wichita.edu  Fri Nov 14 00:29:17 2008
From: rodney.bates at wichita.edu (Rodney M. Bates)
Date: Thu, 13 Nov 2008 17:29:17 -0600
Subject: [M3devel] The undefined ROOT problem, rebuilding cm3
Message-ID: <491CB84D.4000008@wichita.edu>

Here is how the undefined ROOT problem happens when trying to rebuild
cm3 with itself.

I had been making the assumption that commands build, ship, clean, when
supplied to the do-cm3-*.sh scripts, did the same things as -build,
-ship, -clean supplied to the cm3 command.  Not so.

build actually does cm3 -build -override, as do buildlocal and empty
commands to the scripts. clean does cm3 -clean -override, as does cleanlocal.
There is a buildglobal command that does -build without -override, but
it also does a ship as well.  There appears to be no way to just build,
without either overriding or shipping.

So, I have been running do-cm3-*.sh with no argument (and sometimes build).
This starts a chain of events that eventually leads to an installed libm3
that always executes quake script bundle-ov.tmpl, and it requires that
either M3BUNDLE or ROOT be defined, neither of which normally is.  This
will happen trying to compile any program that uses libm3, which is just
about everything.

We should document this somewhere.


From jay.krell at cornell.edu  Fri Nov 14 02:44:44 2008
From: jay.krell at cornell.edu (Jay)
Date: Fri, 14 Nov 2008 01:44:44 +0000
Subject: [M3devel] The undefined ROOT problem, rebuilding cm3
In-Reply-To: <491CB84D.4000008@wichita.edu>
References: <491CB84D.4000008@wichita.edu>
Message-ID: <COL101-W79AA2376232A9B60AC3D33E6160@phx.gbl>


I don't know, but something doesn't seem right.
 
D:\cm3>dir /s/b bund* *ov* *temp* *tmp*...
D:\cm3\pkg\libm3\src\bundleintf\bundle.tmpl
 
if defined("M3BUNDLE")  _M3BUNDLE = M3BUNDLEelse  _M3BUNDLE = BIN_USE & SL & "m3bundle"end
D:\dev2\cm3.2>dir /s/b /a-d bund*tmp*D:\dev2\cm3.2\doc\help\gen_html\libm3\src\bundleintf\bundle.tmpl.htmlD:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle-ov.tmplD:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle.tmpl
 
:\dev2\cm3.2>dir /s/b /a-d bund*tmp*:\dev2\cm3.2\doc\help\gen_html\libm3\src\bundleintf\bundle.tmpl.html:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle-ov.tmpl:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle.tmpl
 
bundle-ov.tmp:
 
if defined("M3BUNDLE")  _M3BUNDLE = M3BUNDLEelse  _M3BUNDLE = format(ROOT & SL & "m3-tools" & SL & "m3bundle" & SL & "%s" &                     SL & "m3bundle", BUILD_DIR)  M3BUNDLE = _M3BUNDLEend
 => this file requires ROOT or M3BUNDLE like you say, but this isn't normally used.
 
D:\dev2\cm3.2>dir /s/b m3makefile | findstr /i bundleintfD:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\m3makefile
 
if defined("_M3BUNDLE_OVERRIDE")  template  ("bundle-ov")endtemplate  ("bundle")
You defined _M3BUNDLE_OVERRIDE somewhere along the way?
 
Try replacing the one file with the other..
 
 - Jay> Date: Thu, 13 Nov 2008 17:29:17 -0600> From: rodney.bates at wichita.edu> To: m3devel at elegosoft.com> Subject: [M3devel] The undefined ROOT problem, rebuilding cm3> > Here is how the undefined ROOT problem happens when trying to rebuild> cm3 with itself.> > I had been making the assumption that commands build, ship, clean, when> supplied to the do-cm3-*.sh scripts, did the same things as -build,> -ship, -clean supplied to the cm3 command. Not so.> > build actually does cm3 -build -override, as do buildlocal and empty> commands to the scripts. clean does cm3 -clean -override, as does cleanlocal.> There is a buildglobal command that does -build without -override, but> it also does a ship as well. There appears to be no way to just build,> without either overriding or shipping.> > So, I have been running do-cm3-*.sh with no argument (and sometimes build).> This starts a chain of events that eventually leads to an installed libm3> that always executes quake script bundle-ov.tmpl, and it requires that> either M3BUNDLE or ROOT be defined, neither of which normally is. This> will happen trying to compile any program that uses libm3, which is just> about everything.> > We should document this somewhere.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081114/3c45491f/attachment-0002.html>

From rodney.bates at wichita.edu  Fri Nov 14 03:01:30 2008
From: rodney.bates at wichita.edu (Rodney M. Bates)
Date: Thu, 13 Nov 2008 20:01:30 -0600
Subject: [M3devel] The undefined ROOT problem, rebuilding cm3
In-Reply-To: <COL101-W79AA2376232A9B60AC3D33E6160@phx.gbl>
References: <491CB84D.4000008@wichita.edu>
	<COL101-W79AA2376232A9B60AC3D33E6160@phx.gbl>
Message-ID: <491CDBFA.7030500@wichita.edu>

Jay wrote:
> I don't know, but something doesn't seem right.
>  
> D:\cm3>dir /s/b bund* *ov* *temp* *tmp*
> ...
> D:\cm3\pkg\libm3\src\bundleintf\bundle.tmpl
>  
> if defined("M3BUNDLE")
>   _M3BUNDLE = M3BUNDLE
> else
>   _M3BUNDLE = BIN_USE & SL & "m3bundle"
> end
> 
> D:\dev2\cm3.2>dir /s/b /a-d bund*tmp*
> D:\dev2\cm3.2\doc\help\gen_html\libm3\src\bundleintf\bundle.tmpl.html
> D:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle-ov.tmpl
> D:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle.tmpl
>  
> :\dev2\cm3.2>dir /s/b /a-d bund*tmp*
> :\dev2\cm3.2\doc\help\gen_html\libm3\src\bundleintf\bundle.tmpl.html
> :\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle-ov.tmpl
> :\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle.tmpl
>  
> bundle-ov.tmp:
>  
> if defined("M3BUNDLE")
>   _M3BUNDLE = M3BUNDLE
> else
>   _M3BUNDLE = format(ROOT & SL & "m3-tools" & SL & "m3bundle" & SL & "%s" &
>                      SL & "m3bundle", BUILD_DIR)
>   M3BUNDLE = _M3BUNDLE
> end
> 
>  => this file requires ROOT or M3BUNDLE like you say, but this isn't 
> normally used.
>  
> D:\dev2\cm3.2>dir /s/b m3makefile | findstr /i bundleintf
> D:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\m3makefile
>  
> if defined("_M3BUNDLE_OVERRIDE")
>   template  ("bundle-ov")
> end
> template  ("bundle")
> 
> You defined _M3BUNDLE_OVERRIDE somewhere along the way?

Maybe I should have given more detail on the chain of events.
You got most of it.  The missing piece is:

m3-libs/libm3/src/m3overrides contains:
_M3BUNDLE_OVERRIDE = "T"

So, 'do-cm3-*.sh build' runs 'cm3 -build -override', causing the
above file of quake code to be interpreted, defining _M3BUNDLE_OVERRIDE,
and the rest is as you have noted.

>  
> Try replacing the one file with the other..
>  
>  - Jay
> 
> 
> 
>  > Date: Thu, 13 Nov 2008 17:29:17 -0600
>  > From: rodney.bates at wichita.edu
>  > To: m3devel at elegosoft.com
>  > Subject: [M3devel] The undefined ROOT problem, rebuilding cm3
>  >
>  > Here is how the undefined ROOT problem happens when trying to rebuild
>  > cm3 with itself.
>  >
>  > I had been making the assumption that commands build, ship, clean, when
>  > supplied to the do-cm3-*.sh scripts, did the same things as -build,
>  > -ship, -clean supplied to the cm3 command. Not so.
>  >
>  > build actually does cm3 -build -override, as do buildlocal and empty
>  > commands to the scripts. clean does cm3 -clean -override, as does 
> cleanlocal.
>  > There is a buildglobal command that does -build without -override, but
>  > it also does a ship as well. There appears to be no way to just build,
>  > without either overriding or shipping.
>  >
>  > So, I have been running do-cm3-*.sh with no argument (and sometimes 
> build).
>  > This starts a chain of events that eventually leads to an installed libm3
>  > that always executes quake script bundle-ov.tmpl, and it requires that
>  > either M3BUNDLE or ROOT be defined, neither of which normally is. This
>  > will happen trying to compile any program that uses libm3, which is just
>  > about everything.
>  >
>  > We should document this somewhere.
> 



From hosking at cs.purdue.edu  Fri Nov 14 03:50:07 2008
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Thu, 13 Nov 2008 20:50:07 -0600
Subject: [M3devel] The undefined ROOT problem, rebuilding cm3
In-Reply-To: <491CDBFA.7030500@wichita.edu>
References: <491CB84D.4000008@wichita.edu>
	<COL101-W79AA2376232A9B60AC3D33E6160@phx.gbl>
	<491CDBFA.7030500@wichita.edu>
Message-ID: <74C195D7-3B89-4696-AB57-DE41DDD09E6D@cs.purdue.edu>

I generally run with buildship for these commands.

On 13 Nov 2008, at 20:01, Rodney M. Bates wrote:

> Jay wrote:
>> I don't know, but something doesn't seem right.
>> D:\cm3>dir /s/b bund* *ov* *temp* *tmp*
>> ...
>> D:\cm3\pkg\libm3\src\bundleintf\bundle.tmpl
>> if defined("M3BUNDLE")
>>  _M3BUNDLE = M3BUNDLE
>> else
>>  _M3BUNDLE = BIN_USE & SL & "m3bundle"
>> end
>> D:\dev2\cm3.2>dir /s/b /a-d bund*tmp*
>> D:\dev2\cm3.2\doc\help\gen_html\libm3\src\bundleintf\bundle.tmpl.html
>> D:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle-ov.tmpl
>> D:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle.tmpl
>> :\dev2\cm3.2>dir /s/b /a-d bund*tmp*
>> :\dev2\cm3.2\doc\help\gen_html\libm3\src\bundleintf\bundle.tmpl.html
>> :\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle-ov.tmpl
>> :\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle.tmpl
>> bundle-ov.tmp:
>> if defined("M3BUNDLE")
>>  _M3BUNDLE = M3BUNDLE
>> else
>>  _M3BUNDLE = format(ROOT & SL & "m3-tools" & SL & "m3bundle" & SL &  
>> "%s" &
>>                     SL & "m3bundle", BUILD_DIR)
>>  M3BUNDLE = _M3BUNDLE
>> end
>> => this file requires ROOT or M3BUNDLE like you say, but this isn't  
>> normally used.
>> D:\dev2\cm3.2>dir /s/b m3makefile | findstr /i bundleintf
>> D:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\m3makefile
>> if defined("_M3BUNDLE_OVERRIDE")
>>  template  ("bundle-ov")
>> end
>> template  ("bundle")
>> You defined _M3BUNDLE_OVERRIDE somewhere along the way?
>
> Maybe I should have given more detail on the chain of events.
> You got most of it.  The missing piece is:
>
> m3-libs/libm3/src/m3overrides contains:
> _M3BUNDLE_OVERRIDE = "T"
>
> So, 'do-cm3-*.sh build' runs 'cm3 -build -override', causing the
> above file of quake code to be interpreted, defining  
> _M3BUNDLE_OVERRIDE,
> and the rest is as you have noted.
>
>> Try replacing the one file with the other..
>>  - Jay
>> > Date: Thu, 13 Nov 2008 17:29:17 -0600
>> > From: rodney.bates at wichita.edu
>> > To: m3devel at elegosoft.com
>> > Subject: [M3devel] The undefined ROOT problem, rebuilding cm3
>> >
>> > Here is how the undefined ROOT problem happens when trying to  
>> rebuild
>> > cm3 with itself.
>> >
>> > I had been making the assumption that commands build, ship,  
>> clean, when
>> > supplied to the do-cm3-*.sh scripts, did the same things as -build,
>> > -ship, -clean supplied to the cm3 command. Not so.
>> >
>> > build actually does cm3 -build -override, as do buildlocal and  
>> empty
>> > commands to the scripts. clean does cm3 -clean -override, as does  
>> cleanlocal.
>> > There is a buildglobal command that does -build without - 
>> override, but
>> > it also does a ship as well. There appears to be no way to just  
>> build,
>> > without either overriding or shipping.
>> >
>> > So, I have been running do-cm3-*.sh with no argument (and  
>> sometimes build).
>> > This starts a chain of events that eventually leads to an  
>> installed libm3
>> > that always executes quake script bundle-ov.tmpl, and it requires  
>> that
>> > either M3BUNDLE or ROOT be defined, neither of which normally is.  
>> This
>> > will happen trying to compile any program that uses libm3, which  
>> is just
>> > about everything.
>> >
>> > We should document this somewhere.



From wagner at elegosoft.com  Fri Nov 14 09:35:07 2008
From: wagner at elegosoft.com (Olaf Wagner)
Date: Fri, 14 Nov 2008 09:35:07 +0100
Subject: [M3devel] The undefined ROOT problem, rebuilding cm3
In-Reply-To: <491CB84D.4000008@wichita.edu>
References: <491CB84D.4000008@wichita.edu>
Message-ID: <20081114093507.rp91pc17kg8cc8gw@mail.elegosoft.com>

Quoting "Rodney M. Bates" <rodney.bates at wichita.edu>:

> Here is how the undefined ROOT problem happens when trying to rebuild
> cm3 with itself.
>
> I had been making the assumption that commands build, ship, clean, when
> supplied to the do-cm3-*.sh scripts, did the same things as -build,
> -ship, -clean supplied to the cm3 command.  Not so.
>
> build actually does cm3 -build -override, as do buildlocal and empty
> commands to the scripts. clean does cm3 -clean -override, as does cleanlocal.
> There is a buildglobal command that does -build without -override, but
> it also does a ship as well.  There appears to be no way to just build,
> without either overriding or shipping.
>
> So, I have been running do-cm3-*.sh with no argument (and sometimes build).
> This starts a chain of events that eventually leads to an installed libm3
> that always executes quake script bundle-ov.tmpl, and it requires that
> either M3BUNDLE or ROOT be defined, neither of which normally is.  This
> will happen trying to compile any program that uses libm3, which is just
> about everything.
>
> We should document this somewhere.

Perhaps you could write up some paragraphs for the scripts' README
and the web pages?
I always seem to overlook or tend to obscure some crucial points for
other users :-/

Olaf
-- 
Olaf Wagner -- elego Software Solutions GmbH
                Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany
phone: +49 30 23 45 86 96  mobile: +49 177 2345 869  fax: +49 30 23 45 86 95
    http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin
Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194



From jay.krell at cornell.edu  Fri Nov 14 12:08:17 2008
From: jay.krell at cornell.edu (Jay)
Date: Fri, 14 Nov 2008 11:08:17 +0000
Subject: [M3devel] The undefined ROOT problem, rebuilding cm3
In-Reply-To: <74C195D7-3B89-4696-AB57-DE41DDD09E6D@cs.purdue.edu>
References: <491CB84D.4000008@wichita.edu>
	<COL101-W79AA2376232A9B60AC3D33E6160@phx.gbl>
	<491CDBFA.7030500@wichita.edu>
	<74C195D7-3B89-4696-AB57-DE41DDD09E6D@cs.purdue.edu>
Message-ID: <COL101-W32D8D3B50ACB900B343094E6160@phx.gbl>


> From: hosking>
> I generally run with buildship for these commands.
 
That, plus if you "buildlocal", you are supposed to be barred from "shipping".
That is, if you build with -override, you aren't supposed to be able to "ship".
"ship" is to copy from the source tree to the install tree.
I think I did see a problem once where it let me ship "local" stuff.
 
 - Jay
 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081114/c160ff55/attachment-0002.html>

From rodney.bates at wichita.edu  Fri Nov 14 16:26:05 2008
From: rodney.bates at wichita.edu (Rodney M. Bates)
Date: Fri, 14 Nov 2008 09:26:05 -0600
Subject: [M3devel] The undefined ROOT problem, rebuilding cm3
In-Reply-To: <74C195D7-3B89-4696-AB57-DE41DDD09E6D@cs.purdue.edu>
References: <491CB84D.4000008@wichita.edu>
	<COL101-W79AA2376232A9B60AC3D33E6160@phx.gbl>
	<491CDBFA.7030500@wichita.edu>
	<74C195D7-3B89-4696-AB57-DE41DDD09E6D@cs.purdue.edu>
Message-ID: <491D988D.9080804@wichita.edu>

Tony Hosking wrote:
> I generally run with buildship for these commands.

And so will I, henceforth.

Rodney Bates




From jay.krell at cornell.edu  Fri Nov 14 21:24:55 2008
From: jay.krell at cornell.edu (Jay)
Date: Fri, 14 Nov 2008 20:24:55 +0000
Subject: [M3devel] Linux/amd64
Message-ID: <COL101-W20B494FEE8697CAC58E0DEE6160@phx.gbl>


I turned of gc just while running stubgen.
 
Leading to:
  http://m3.elegosoft.com/cm3/uploaded-archives/cm3-std-POSIX-AMD64_LINUX-d5.7.0.tar.gz  
  http://m3.elegosoft.com/cm3/uploaded-archives/cm3-min-POSIX-AMD64_LINUX-d5.7.0.tar.gz  

Please try them? See how well they work for you all?
I haven't run most of the stuff, but cm3 does build itself.
gc does usually?often work.

(saying "posix" I feel is redundant...)

 - Jay

From jay.krell at cornell.edu  Sun Nov 16 16:31:25 2008
From: jay.krell at cornell.edu (Jay)
Date: Sun, 16 Nov 2008 15:31:25 +0000
Subject: [M3devel] duplicated code?
Message-ID: <COL101-W82BC319326E633D918F12DE6100@phx.gbl>


Anyone want to clean up this duplicity?
 
 
D:\dev2\cm3.2>dir /s/b asttotype.m3  StubCode.m3D:\dev2\cm3.2\m3-comm\sharedobjgen\src\AstToType.m3D:\dev2\cm3.2\m3-comm\stubgen\src\AstToType.m3D:\dev2\cm3.2\m3-db\stablegen\src\AstToType.m3
D:\dev2\cm3.2\m3-comm\sharedobjgen\src\StubCode.m3D:\dev2\cm3.2\m3-comm\stubgen\src\StubCode.m3
 
somewhere in there, AMD64_LINUX tries to allocate a lot of memory, and fails
either there or soon thereafter.
The garbage collector is working.
 
 - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081116/deecc4a6/attachment-0002.html>

From jay.krell at cornell.edu  Sun Nov 16 17:18:40 2008
From: jay.krell at cornell.edu (Jay)
Date: Sun, 16 Nov 2008 16:18:40 +0000
Subject: [M3devel] duplicated code?
In-Reply-To: <COL101-W82BC319326E633D918F12DE6100@phx.gbl>
References: <COL101-W82BC319326E633D918F12DE6100@phx.gbl>
Message-ID: <COL101-W16C880C91AE709A4E8463FE6100@phx.gbl>


Er, the large allocation does come from the gc itself.
 
Tony, is there is an assumption that the heap is contiguous?That calls to RTOS.GetMemory return adjacent addresses?
 
This code allocates a large array:
 
    IF desc = NIL OR newSideSpan # NUMBER(desc^) THEN      WITH newDesc = NEW(UNTRACED REF ARRAY OF Desc, newSideSpan) DO
I'll know more shortly.
 
I guess..it looks like the heap can be discontiguous, but
we do record keeping for what it all spans.
 
The comments say:
(* The array desc and the global variables p0, and p1 describe the pages   that are part of the traced heap.  Either p0 and p1 are equal to Nil and   no pages are allocated; or both are valid pages and page p is allocated   iff
|          p0 <= p < p1|      AND desc[p - p0] != Unallocated
 
Hm..
 
Grow (0x44000) => 0x2b1e45256000   total: 1.5M
GetUntracedOpenArray(0x1a80)   span: 6.6M   density: 24%stubgen: Processing RemoteView.TGetUntracedOpenArray(0x3f0)t1:0xct2:0xat3:0x1Grow (0x52000) => 0x2aaaaaaab000   total: 1.8M
GetUntracedOpenArray(0x1ce69fc8)
 
I have GetUntracedOpenArray printing how many bytes it is asked for.
t1,t2,t3 are just the lengths of the strings being concatented.
Grow(x)=>y means Grow allocated x bytes at address y.
 
So now, these two addresses 0x2aaaaaaab000 and 0x2b1e45256000  are very far apart, like 400gig.
And it seems the heap allocator wants to allocate an array to describe the pages.
 
Hm. Page size is no longer tied to the underlying system -- no longer vm-tied gc.
Perhaps perhaps blowing it up, to say, 1meg, will address this?
 
But really, an array to describe pages spanning the results of separate memory allocations, seems wrong.
A sparser data structure would be good, that could describe arbitrary sized runs of pages as being in the same state.
 
 - Jay



From: jay.krell at cornell.eduTo: m3devel at elegosoft.comDate: Sun, 16 Nov 2008 15:31:25 +0000Subject: [M3devel] duplicated code?

Anyone want to clean up this duplicity?  D:\dev2\cm3.2>dir /s/b asttotype.m3  StubCode.m3D:\dev2\cm3.2\m3-comm\sharedobjgen\src\AstToType.m3D:\dev2\cm3.2\m3-comm\stubgen\src\AstToType.m3D:\dev2\cm3.2\m3-db\stablegen\src\AstToType.m3D:\dev2\cm3.2\m3-comm\sharedobjgen\src\StubCode.m3D:\dev2\cm3.2\m3-comm\stubgen\src\StubCode.m3 somewhere in there, AMD64_LINUX tries to allocate a lot of memory, and failseither there or soon thereafter.The garbage collector is working.  - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081116/a28dfe37/attachment-0002.html>

From hosking at cs.purdue.edu  Mon Nov 17 03:14:55 2008
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Sun, 16 Nov 2008 20:14:55 -0600
Subject: [M3devel] duplicated code?
In-Reply-To: <COL101-W16C880C91AE709A4E8463FE6100@phx.gbl>
References: <COL101-W82BC319326E633D918F12DE6100@phx.gbl>
	<COL101-W16C880C91AE709A4E8463FE6100@phx.gbl>
Message-ID: <4B6BDC53-DB07-4498-A0D4-CB383DFFD487@cs.purdue.edu>

On 16 Nov 2008, at 10:18, Jay wrote:

> Er, the large allocation does come from the gc itself.

What's being allocated?  The side array should not be too huge.

> Tony, is there is an assumption that the heap is contiguous?
> That calls to RTOS.GetMemory return adjacent addresses?

No assumption.

> This code allocates a large array:
>
>     IF desc = NIL OR newSideSpan # NUMBER(desc^) THEN
>       WITH newDesc = NEW(UNTRACED REF ARRAY OF Desc, newSideSpan) DO
>
> I'll know more shortly.
>
> I guess..it looks like the heap can be discontiguous, but
> we do record keeping for what it all spans.

Correct.

> The comments say:
> (* The array desc and the global variables p0, and p1 describe the  
> pages
>    that are part of the traced heap.  Either p0 and p1 are equal to  
> Nil and
>    no pages are allocated; or both are valid pages and page p is  
> allocated
>    iff
> |          p0 <= p < p1
> |      AND desc[p - p0] != Unallocated
>
>
> Hm..
>
> Grow (0x44000) => 0x2b1e45256000   total: 1.5M
> GetUntracedOpenArray(0x1a80)
>    span: 6.6M   density: 24%
> stubgen: Processing RemoteView.T
> GetUntracedOpenArray(0x3f0)
> t1:0xc
> t2:0xa
> t3:0x1
> Grow (0x52000) => 0x2aaaaaaab000   total: 1.8M
> GetUntracedOpenArray(0x1ce69fc8)
>
> I have GetUntracedOpenArray printing how many bytes it is asked for.
> t1,t2,t3 are just the lengths of the strings being concatented.
> Grow(x)=>y means Grow allocated x bytes at address y.
>
> So now, these two addresses 0x2aaaaaaab000 and 0x2b1e45256000  are  
> very far apart, like 400gig.
> And it seems the heap allocator wants to allocate an array to  
> describe the pages.

Ah, yes, that is most unfortunate.


> Hm. Page size is no longer tied to the underlying system -- no  
> longer vm-tied gc.
> Perhaps perhaps blowing it up, to say, 1meg, will address this?

I am working on minimizing the need for the global array, but we do  
need something that can be easily indexed like this.  Perhaps we need  
to pass a hint to RTOS.GetMemory that will try to allocate its regions  
close together.

> But really, an array to describe pages spanning the results of  
> separate memory allocations, seems wrong.
> A sparser data structure would be good, that could describe  
> arbitrary sized runs of pages as being in the same state.

Indeed.  As mentioned above, I am working to eliminate the need for  
this.  Code that starts us on this path will be checked in within a  
day or so.

>  - Jay
>
>
>
> From: jay.krell at cornell.edu
> To: m3devel at elegosoft.com
> Date: Sun, 16 Nov 2008 15:31:25 +0000
> Subject: [M3devel] duplicated code?
>
> Anyone want to clean up this duplicity?
>
>
> D:\dev2\cm3.2>dir /s/b asttotype.m3  StubCode.m3
> D:\dev2\cm3.2\m3-comm\sharedobjgen\src\AstToType.m3
> D:\dev2\cm3.2\m3-comm\stubgen\src\AstToType.m3
> D:\dev2\cm3.2\m3-db\stablegen\src\AstToType.m3
> D:\dev2\cm3.2\m3-comm\sharedobjgen\src\StubCode.m3
> D:\dev2\cm3.2\m3-comm\stubgen\src\StubCode.m3
>
>
> somewhere in there, AMD64_LINUX tries to allocate a lot of memory,  
> and fails
> either there or soon thereafter.
> The garbage collector is working.
>
>  - Jay
>
>
>
>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081116/e6f957c5/attachment-0002.html>

From jay.krell at cornell.edu  Mon Nov 17 09:39:48 2008
From: jay.krell at cornell.edu (Jay)
Date: Mon, 17 Nov 2008 08:39:48 +0000
Subject: [M3devel] AMD64_LINUX (again)
Message-ID: <COL101-W10D65D8CA3093DB21EB5D1E6130@phx.gbl>


ok, with the sbrk workaround now:
 
 
http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-std-AMD64_LINUX-d5.7.0.tar.gz
 You don't have to build anything.
 
http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-min-AMD64_LINUX-d5.7.0.tar.gz
  You probably want to run ./do-cm3-std.py buildship.
 
http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-AMD64_LINUX.gz
  You get to build your own of almost everything (./install-config.py && ./do-pkg.py m3cc buildship && ./do-cm3-front.py buildship && ./install-front.py && ./do-cm3-std.py buildship)
 
 
OR you can bootstrap from an existing other system, roughly:
 cd m3-sys/m3cc && cm3 -DM3CC_TARGET=AMD64_LINUX && cd scripts/python && ./do-cm3-front.py AMD64_LINUX boot && ./boot1.py AMD64_LINUX and copy /cm3-boot-AMD64_LINUX-1.tar.gz to target machine and extract it, cd into it, make, and then you have native cm3 and proceed to previous..
 
 
I'm able to run Juno, Cube, Calculator, Mentor.
 
Hm. formsedit crashes. Darn.
 
Program received signal SIGSEGV, Segmentation fault.[Switching to Thread 1140881760 (LWP 13270)]0x00002afc180b9d6a in ScrollerVBTClass__PaintViewWithShadows (M3_Ao3CED_v=Cannot access memory at address 0x144006bf8)    at ../src/lego/POSIX/ScrollerVBTClass.m3:325325         VBT.PaintTint(v, r, v.shadow.bg);(gdb) cContinuing.Cannot find user-level thread for LWP 13242: generic error(gdb) bt#0  0x00002afc180b9d6a in ScrollerVBTClass__PaintViewWithShadows (M3_Ao3CED_v=Cannot access memory at address 0x144006bf8
) at ../src/lego/POSIX/ScrollerVBTClass.m3:325Cannot fetch general-purpose registers for thread 1140881760: generic error(gdb)
 
 
still might be worth other people trying out..
 
 
 - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081117/d6aecb25/attachment-0002.html>

From wagner at elegosoft.com  Mon Nov 17 10:10:43 2008
From: wagner at elegosoft.com (Olaf Wagner)
Date: Mon, 17 Nov 2008 10:10:43 +0100
Subject: [M3devel] The undefined ROOT problem, rebuilding cm3
In-Reply-To: <491CB84D.4000008@wichita.edu>
References: <491CB84D.4000008@wichita.edu>
Message-ID: <20081117101043.m08cyzrq8gkksco8@mail.elegosoft.com>

Quoting "Rodney M. Bates" <rodney.bates at wichita.edu>:

> Here is how the undefined ROOT problem happens when trying to rebuild
> cm3 with itself.
>
> I had been making the assumption that commands build, ship, clean, when
> supplied to the do-cm3-*.sh scripts, did the same things as -build,
> -ship, -clean supplied to the cm3 command.  Not so.
>
> build actually does cm3 -build -override, as do buildlocal and empty
> commands to the scripts. clean does cm3 -clean -override, as does cleanlocal.
> There is a buildglobal command that does -build without -override, but
> it also does a ship as well.  There appears to be no way to just build,
> without either overriding or shipping.
>
> So, I have been running do-cm3-*.sh with no argument (and sometimes build).
> This starts a chain of events that eventually leads to an installed libm3
> that always executes quake script bundle-ov.tmpl, and it requires that
> either M3BUNDLE or ROOT be defined, neither of which normally is.  This
> will happen trying to compile any program that uses libm3, which is just
> about everything.
>
> We should document this somewhere.

It seems that this time my email wasn't complete, for whatever reasons.
I was out of town for some days so I noticed only today.

Rodney, I did ask you if you could write up some paragraphs for the
README in scripts and the web pages that clarifies this matter so that
others won't run into it again?

I always seem to miss some crucial points when trying to explain the
build system myself...

Thanks in advance,

Olaf
-- 
Olaf Wagner -- elego Software Solutions GmbH
                Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany
phone: +49 30 23 45 86 96  mobile: +49 177 2345 869  fax: +49 30 23 45 86 95
    http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin
Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194



From wagner at elegosoft.com  Mon Nov 17 10:30:00 2008
From: wagner at elegosoft.com (Olaf Wagner)
Date: Mon, 17 Nov 2008 10:30:00 +0100
Subject: [M3devel] The undefined ROOT problem, rebuilding cm3
In-Reply-To: <491CB84D.4000008@wichita.edu>
References: <491CB84D.4000008@wichita.edu>
Message-ID: <20081117103000.ld2m2zsu8w48o4ck@mail.elegosoft.com>

Hm, now I seem to be the victim of mail truncation. I'll  try a top-quote
this time (our system administrator will investigate this). Third attempt:

It seems that this time my email wasn't complete, for whatever reasons.
I was out of town for some days so I noticed only today.

Rodney, I did ask you if you could write up some paragraphs for the
README in scripts and the web pages that clarifies this matter so that
others won't run into it again?

I always seem to miss some crucial points when trying to explain the
build system myself...

Thanks in advance,

Olaf

Quoting "Rodney M. Bates" <rodney.bates at wichita.edu>:

> Here is how the undefined ROOT problem happens when trying to rebuild
> cm3 with itself.
>
> I had been making the assumption that commands build, ship, clean, when
> supplied to the do-cm3-*.sh scripts, did the same things as -build,
> -ship, -clean supplied to the cm3 command.  Not so.
>
> build actually does cm3 -build -override, as do buildlocal and empty
> commands to the scripts. clean does cm3 -clean -override, as does cleanlocal.
> There is a buildglobal command that does -build without -override, but
> it also does a ship as well.  There appears to be no way to just build,
> without either overriding or shipping.
>
> So, I have been running do-cm3-*.sh with no argument (and sometimes build).
> This starts a chain of events that eventually leads to an installed libm3
> that always executes quake script bundle-ov.tmpl, and it requires that
> either M3BUNDLE or ROOT be defined, neither of which normally is.  This
> will happen trying to compile any program that uses libm3, which is just
> about everything.
>
> We should document this somewhere.



-- 
Olaf Wagner -- elego Software Solutions GmbH
                Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany
phone: +49 30 23 45 86 96  mobile: +49 177 2345 869  fax: +49 30 23 45 86 95
    http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin
Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194



From jay.krell at cornell.edu  Mon Nov 17 10:35:18 2008
From: jay.krell at cornell.edu (Jay)
Date: Mon, 17 Nov 2008 09:35:18 +0000
Subject: [M3devel] gc vs. large spans of memory
In-Reply-To: <4B6BDC53-DB07-4498-A0D4-CB383DFFD487@cs.purdue.edu>
References: <COL101-W82BC319326E633D918F12DE6100@phx.gbl>
	<COL101-W16C880C91AE709A4E8463FE6100@phx.gbl> 
	<4B6BDC53-DB07-4498-A0D4-CB383DFFD487@cs.purdue.edu>
Message-ID: <BAY120-W65F83A8674A6D4949CA7CE6130@phx.gbl>


1meg pages weren't large enough to avoid running out of memory.
 
 
Any thoughts on using the boehm-gc?
I don't know if it is compacting.
Or even generational.
Or anything about it really, just that it is much ported and presumably well maintained, since it is part of "gcc", part of the gcc Java support.
 
 
Hm... I assume the right data structure here is a multi level array, that you index by picking off progressively less significant bits of an address (or page number) just like the hardware page tables for virtual memory (but without the perf benefits of a cache and using physical addresses after the first level).
When an entire level is at the same state, due to skipping a large span, you just have one shared entry for all of them, major savings.
 
 
Either that, or a binary searched array where you only store entries for state changes.
You'd use like the STL's upper_bound/lower_bound/equal_range functions, that is, really the same thing as bsearch, but when an entry isn't found, you return where it would be inserted, which is like the last place you looked before giving up.
 
It might be worth confirming "that I'm not crazy" -- that mmap behaves this way for other people, or that skimming its code suggests it is not surprising. Could be related to how malloc works also.
 
 - Jay



From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 16 Nov 2008 20:14:55 -0600CC: m3devel at elegosoft.comSubject: Re: [M3devel] duplicated code?


On 16 Nov 2008, at 10:18, Jay wrote:


Er, the large allocation does come from the gc itself.

What's being allocated?  The side array should not be too huge.

Tony, is there is an assumption that the heap is contiguous?That calls to RTOS.GetMemory return adjacent addresses?

No assumption.

This code allocates a large array:     IF desc = NIL OR newSideSpan # NUMBER(desc^) THEN      WITH newDesc = NEW(UNTRACED REF ARRAY OF Desc, newSideSpan) DOI'll know more shortly. I guess..it looks like the heap can be discontiguous, butwe do record keeping for what it all spans.

Correct.

The comments say:(* The array desc and the global variables p0, and p1 describe the pages   that are part of the traced heap.  Either p0 and p1 are equal to Nil and   no pages are allocated; or both are valid pages and page p is allocated   iff|          p0 <= p < p1|      AND desc[p - p0] != Unallocated Hm.. Grow (0x44000) => 0x2b1e45256000   total: 1.5MGetUntracedOpenArray(0x1a80)   span: 6.6M   density: 24%stubgen: Processing RemoteView.TGetUntracedOpenArray(0x3f0)t1:0xct2:0xat3:0x1Grow (0x52000) => 0x2aaaaaaab000   total: 1.8MGetUntracedOpenArray(0x1ce69fc8) I have GetUntracedOpenArray printing how many bytes it is asked for.t1,t2,t3 are just the lengths of the strings being concatented.Grow(x)=>y means Grow allocated x bytes at address y. So now, these two addresses 0x2aaaaaaab000 and 0x2b1e45256000  are very far apart, like 400gig.And it seems the heap allocator wants to allocate an array to describe the pages.

Ah, yes, that is most unfortunate.


Hm. Page size is no longer tied to the underlying system -- no longer vm-tied gc.Perhaps perhaps blowing it up, to say, 1meg, will address this?

I am working on minimizing the need for the global array, but we do need something that can be easily indexed like this.  Perhaps we need to pass a hint to RTOS.GetMemory that will try to allocate its regions close together.

But really, an array to describe pages spanning the results of separate memory allocations, seems wrong.A sparser data structure would be good, that could describe arbitrary sized runs of pages as being in the same state.

Indeed.  As mentioned above, I am working to eliminate the need for this.  Code that starts us on this path will be checked in within a day or so.

 - Jay

From: jay.krell at cornell.eduTo: m3devel at elegosoft.comDate: Sun, 16 Nov 2008 15:31:25 +0000Subject: [M3devel] duplicated code?Anyone want to clean up this duplicity?  D:\dev2\cm3.2>dir /s/b asttotype.m3  StubCode.m3D:\dev2\cm3.2\m3-comm\sharedobjgen\src\AstToType.m3D:\dev2\cm3.2\m3-comm\stubgen\src\AstToType.m3D:\dev2\cm3.2\m3-db\stablegen\src\AstToType.m3D:\dev2\cm3.2\m3-comm\sharedobjgen\src\StubCode.m3D:\dev2\cm3.2\m3-comm\stubgen\src\StubCode.m3 somewhere in there, AMD64_LINUX tries to allocate a lot of memory, and failseither there or soon thereafter.The garbage collector is working.  - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081117/8e6723b0/attachment-0002.html>

From hosking at cs.purdue.edu  Mon Nov 17 13:21:26 2008
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Mon, 17 Nov 2008 06:21:26 -0600
Subject: [M3devel] gc vs. large spans of memory
In-Reply-To: <BAY120-W65F83A8674A6D4949CA7CE6130@phx.gbl>
References: <COL101-W82BC319326E633D918F12DE6100@phx.gbl>
	<COL101-W16C880C91AE709A4E8463FE6100@phx.gbl>
	<4B6BDC53-DB07-4498-A0D4-CB383DFFD487@cs.purdue.edu>
	<BAY120-W65F83A8674A6D4949CA7CE6130@phx.gbl>
Message-ID: <AE6DC073-2F37-437A-8712-0C5F09B5A97D@cs.purdue.edu>

On 17 Nov 2008, at 03:35, Jay wrote:

> 1meg pages weren't large enough to avoid running out of memory.

Is this still the sidespan array problem?

> Any thoughts on using the boehm-gc?

Hell no.

> I don't know if it is compacting.

Indeed, Boehm's collector does not do compaction.  Also, Boehm's is  
fully conservative ("ambiguous-roots + ambiguous heap"), though it is  
possible to configure it to use precise heap information (which CM3  
does provide).

> Or even generational.

It is generational, and has a parallel collection mode, but it is not  
concurrent like ours (we have a better chance of scaling on SMPs).   
Our collector will soon also be "on-the-fly", which means we won't  
need to have a stop-the-world phase where all the threads are stopped  
at the same time to initiate GC.  Instead, we will simply signal  
threads one at a time to prepare for GC.

> Or anything about it really, just that it is much ported and  
> presumably well maintained, since it is part of "gcc", part of the  
> gcc Java support.

Our collector is much ported and well-maintained too! ;-)  Seriously,  
I am quite sure we have no collector bugs currently (it has been  
running reliably for many years now).  The problems you are  
encountering are porting issues and not bugs in the collector.  Also,  
the current collector is written in Modula-3, and nicely integrated  
with the Modula-3 object model and run-time.   There is a *huge*  
amount to be said for eating your own dog-food by writing the  
collector in Modula-3.  There is *no* good reason to step outside  
Modula-3 to C as would be needed for Boehm GC.

> Hm... I assume the right data structure here is a multi level array,  
> that you index by picking off progressively less significant bits of  
> an address (or page number) just like the hardware page tables for  
> virtual memory (but without the perf benefits of a cache and using  
> physical addresses after the first level).
> When an entire level is at the same state, due to skipping a large  
> span, you just have one shared entry for all of them, major savings.

Right.  I have this partially implemented already.

> Either that, or a binary searched array where you only store entries  
> for state changes.
> You'd use like the STL's upper_bound/lower_bound/equal_range  
> functions, that is, really the same thing as bsearch, but when an  
> entry isn't found, you return where it would be inserted, which is  
> like the last place you looked before giving up.

I'm concerned about the lookup costs for this.

> It might be worth confirming "that I'm not crazy" -- that mmap  
> behaves this way for other people, or that skimming its code  
> suggests it is not surprising. Could be related to how malloc works  
> also.

Are there hints that can be passed to mmap on that platform to cause  
less scattered mappings?  It is odd that sbrk is less scattered, since  
it must also bottom out at mmap, but perhaps it is trying to maintain  
a reasonably compact allocation below the "brk".

>
>
>  - Jay
>
>
> From: hosking at cs.purdue.edu
> To: jay.krell at cornell.edu
> Date: Sun, 16 Nov 2008 20:14:55 -0600
> CC: m3devel at elegosoft.com
> Subject: Re: [M3devel] duplicated code?
>
>
> On 16 Nov 2008, at 10:18, Jay wrote:
>
> Er, the large allocation does come from the gc itself.
>
> What's being allocated?  The side array should not be too huge.
>
> Tony, is there is an assumption that the heap is contiguous?
> That calls to RTOS.GetMemory return adjacent addresses?
>
> No assumption.
>
> This code allocates a large array:
>
>     IF desc = NIL OR newSideSpan # NUMBER(desc^) THEN
>       WITH newDesc = NEW(UNTRACED REF ARRAY OF Desc, newSideSpan) DO
>
> I'll know more shortly.
>
> I guess..it looks like the heap can be discontiguous, but
> we do record keeping for what it all spans.
>
> Correct.
>
> The comments say:
> (* The array desc and the global variables p0, and p1 describe the  
> pages
>    that are part of the traced heap.  Either p0 and p1 are equal to  
> Nil and
>    no pages are allocated; or both are valid pages and page p is  
> allocated
>    iff
> |          p0 <= p < p1
> |      AND desc[p - p0] != Unallocated
>
>
> Hm..
>
> Grow (0x44000) => 0x2b1e45256000   total: 1.5M
> GetUntracedOpenArray(0x1a80)
>    span: 6.6M   density: 24%
> stubgen: Processing RemoteView.T
> GetUntracedOpenArray(0x3f0)
> t1:0xc
> t2:0xa
> t3:0x1
> Grow (0x52000) => 0x2aaaaaaab000   total: 1.8M
> GetUntracedOpenArray(0x1ce69fc8)
>
> I have GetUntracedOpenArray printing how many bytes it is asked for.
> t1,t2,t3 are just the lengths of the strings being concatented.
> Grow(x)=>y means Grow allocated x bytes at address y.
>
> So now, these two addresses 0x2aaaaaaab000 and 0x2b1e45256000  are  
> very far apart, like 400gig.
> And it seems the heap allocator wants to allocate an array to  
> describe the pages.
>
> Ah, yes, that is most unfortunate.
>
>
> Hm. Page size is no longer tied to the underlying system -- no  
> longer vm-tied gc.
> Perhaps perhaps blowing it up, to say, 1meg, will address this?
>
> I am working on minimizing the need for the global array, but we do  
> need something that can be easily indexed like this.  Perhaps we  
> need to pass a hint to RTOS.GetMemory that will try to allocate its  
> regions close together.
>
> But really, an array to describe pages spanning the results of  
> separate memory allocations, seems wrong.
> A sparser data structure would be good, that could describe  
> arbitrary sized runs of pages as being in the same state.
>
> Indeed.  As mentioned above, I am working to eliminate the need for  
> this.  Code that starts us on this path will be checked in within a  
> day or so.
>
>  - Jay
>
>
>
> From: jay.krell at cornell.edu
> To: m3devel at elegosoft.com
> Date: Sun, 16 Nov 2008 15:31:25 +0000
> Subject: [M3devel] duplicated code?
>
> Anyone want to clean up this duplicity?
>
>
> D:\dev2\cm3.2>dir /s/b asttotype.m3  StubCode.m3
> D:\dev2\cm3.2\m3-comm\sharedobjgen\src\AstToType.m3
> D:\dev2\cm3.2\m3-comm\stubgen\src\AstToType.m3
> D:\dev2\cm3.2\m3-db\stablegen\src\AstToType.m3
> D:\dev2\cm3.2\m3-comm\sharedobjgen\src\StubCode.m3
> D:\dev2\cm3.2\m3-comm\stubgen\src\StubCode.m3
>
>
> somewhere in there, AMD64_LINUX tries to allocate a lot of memory,  
> and fails
> either there or soon thereafter.
> The garbage collector is working.
>
>  - Jay
>
>
>
>
>
>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081117/70cfa516/attachment-0002.html>

From jay.krell at cornell.edu  Mon Nov 17 13:48:59 2008
From: jay.krell at cornell.edu (Jay)
Date: Mon, 17 Nov 2008 12:48:59 +0000
Subject: [M3devel] gc vs. large spans of memory
In-Reply-To: <AE6DC073-2F37-437A-8712-0C5F09B5A97D@cs.purdue.edu>
References: <COL101-W82BC319326E633D918F12DE6100@phx.gbl>
	<COL101-W16C880C91AE709A4E8463FE6100@phx.gbl>
	<4B6BDC53-DB07-4498-A0D4-CB383DFFD487@cs.purdue.edu>
	<BAY120-W65F83A8674A6D4949CA7CE6130@phx.gbl>
	<AE6DC073-2F37-437A-8712-0C5F09B5A97D@cs.purdue.edu>
Message-ID: <COL101-W638E7324891BF366626DA4E6130@phx.gbl>


understood, ok.
 
I don't know the algorithm for allocating pages.
 
There is a "hint" parameter to mmap, it isn't exactly clear how it is handled.
Probably giving the block we have plus its size is reasonable.
I can try that out at some point, though using sbrk seems ok.
 
 - Jay



CC: m3devel at elegosoft.comFrom: hosking at cs.purdue.eduTo: jay.krell at cornell.eduSubject: Re: gc vs. large spans of memoryDate: Mon, 17 Nov 2008 06:21:26 -0600


On 17 Nov 2008, at 03:35, Jay wrote:


1meg pages weren't large enough to avoid running out of memory.

Is this still the sidespan array problem?

Any thoughts on using the boehm-gc?

Hell no.

I don't know if it is compacting.

Indeed, Boehm's collector does not do compaction.  Also, Boehm's is fully conservative ("ambiguous-roots + ambiguous heap"), though it is possible to configure it to use precise heap information (which CM3 does provide).

Or even generational.

It is generational, and has a parallel collection mode, but it is not concurrent like ours (we have a better chance of scaling on SMPs).  Our collector will soon also be "on-the-fly", which means we won't need to have a stop-the-world phase where all the threads are stopped at the same time to initiate GC.  Instead, we will simply signal threads one at a time to prepare for GC.

Or anything about it really, just that it is much ported and presumably well maintained, since it is part of "gcc", part of the gcc Java support.

Our collector is much ported and well-maintained too! ;-)  Seriously, I am quite sure we have no collector bugs currently (it has been running reliably for many years now).  The problems you are encountering are porting issues and not bugs in the collector.  Also, the current collector is written in Modula-3, and nicely integrated with the Modula-3 object model and run-time.   There is a *huge* amount to be said for eating your own dog-food by writing the collector in Modula-3.  There is *no* good reason to step outside Modula-3 to C as would be needed for Boehm GC.

Hm... I assume the right data structure here is a multi level array, that you index by picking off progressively less significant bits of an address (or page number) just like the hardware page tables for virtual memory (but without the perf benefits of a cache and using physical addresses after the first level).When an entire level is at the same state, due to skipping a large span, you just have one shared entry for all of them, major savings.

Right.  I have this partially implemented already.

Either that, or a binary searched array where you only store entries for state changes.You'd use like the STL's upper_bound/lower_bound/equal_range functions, that is, really the same thing as bsearch, but when an entry isn't found, you return where it would be inserted, which is like the last place you looked before giving up.

I'm concerned about the lookup costs for this.

It might be worth confirming "that I'm not crazy" -- that mmap behaves this way for other people, or that skimming its code suggests it is not surprising. Could be related to how malloc works also.

Are there hints that can be passed to mmap on that platform to cause less scattered mappings?  It is odd that sbrk is less scattered, since it must also bottom out at mmap, but perhaps it is trying to maintain a reasonably compact allocation below the "brk".

  - Jay

From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 16 Nov 2008 20:14:55 -0600CC: m3devel at elegosoft.comSubject: Re: [M3devel] duplicated code?


On 16 Nov 2008, at 10:18, Jay wrote:


Er, the large allocation does come from the gc itself.

What's being allocated?  The side array should not be too huge.

Tony, is there is an assumption that the heap is contiguous?That calls to RTOS.GetMemory return adjacent addresses?

No assumption.

This code allocates a large array:     IF desc = NIL OR newSideSpan # NUMBER(desc^) THEN      WITH newDesc = NEW(UNTRACED REF ARRAY OF Desc, newSideSpan) DOI'll know more shortly. I guess..it looks like the heap can be discontiguous, butwe do record keeping for what it all spans.

Correct.

The comments say:(* The array desc and the global variables p0, and p1 describe the pages   that are part of the traced heap.  Either p0 and p1 are equal to Nil and   no pages are allocated; or both are valid pages and page p is allocated   iff|          p0 <= p < p1|      AND desc[p - p0] != Unallocated Hm.. Grow (0x44000) => 0x2b1e45256000   total: 1.5MGetUntracedOpenArray(0x1a80)   span: 6.6M   density: 24%stubgen: Processing RemoteView.TGetUntracedOpenArray(0x3f0)t1:0xct2:0xat3:0x1Grow (0x52000) => 0x2aaaaaaab000   total: 1.8MGetUntracedOpenArray(0x1ce69fc8) I have GetUntracedOpenArray printing how many bytes it is asked for.t1,t2,t3 are just the lengths of the strings being concatented.Grow(x)=>y means Grow allocated x bytes at address y. So now, these two addresses 0x2aaaaaaab000 and 0x2b1e45256000  are very far apart, like 400gig.And it seems the heap allocator wants to allocate an array to describe the pages.

Ah, yes, that is most unfortunate.


Hm. Page size is no longer tied to the underlying system -- no longer vm-tied gc.Perhaps perhaps blowing it up, to say, 1meg, will address this?

I am working on minimizing the need for the global array, but we do need something that can be easily indexed like this.  Perhaps we need to pass a hint to RTOS.GetMemory that will try to allocate its regions close together.

But really, an array to describe pages spanning the results of separate memory allocations, seems wrong.A sparser data structure would be good, that could describe arbitrary sized runs of pages as being in the same state.

Indeed.  As mentioned above, I am working to eliminate the need for this.  Code that starts us on this path will be checked in within a day or so.

 - Jay

From: jay.krell at cornell.eduTo: m3devel at elegosoft.comDate: Sun, 16 Nov 2008 15:31:25 +0000Subject: [M3devel] duplicated code?Anyone want to clean up this duplicity?  D:\dev2\cm3.2>dir /s/b asttotype.m3  StubCode.m3D:\dev2\cm3.2\m3-comm\sharedobjgen\src\AstToType.m3D:\dev2\cm3.2\m3-comm\stubgen\src\AstToType.m3D:\dev2\cm3.2\m3-db\stablegen\src\AstToType.m3D:\dev2\cm3.2\m3-comm\sharedobjgen\src\StubCode.m3D:\dev2\cm3.2\m3-comm\stubgen\src\StubCode.m3 somewhere in there, AMD64_LINUX tries to allocate a lot of memory, and failseither there or soon thereafter.The garbage collector is working.  - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081117/4253a60f/attachment-0002.html>

From jay.krell at cornell.edu  Mon Nov 17 14:00:01 2008
From: jay.krell at cornell.edu (Jay)
Date: Mon, 17 Nov 2008 13:00:01 +0000
Subject: [M3devel] gc vs. large spans of memory
In-Reply-To: <AE6DC073-2F37-437A-8712-0C5F09B5A97D@cs.purdue.edu>
References: <COL101-W82BC319326E633D918F12DE6100@phx.gbl>
	<COL101-W16C880C91AE709A4E8463FE6100@phx.gbl>
	<4B6BDC53-DB07-4498-A0D4-CB383DFFD487@cs.purdue.edu>
	<BAY120-W65F83A8674A6D4949CA7CE6130@phx.gbl>
	<AE6DC073-2F37-437A-8712-0C5F09B5A97D@cs.purdue.edu> 
Message-ID: <COL101-W25E6863F4A5BFE2141AA24E6130@phx.gbl>


 > 1 meg pages didn't suffice
 > Is this still the sidespan array problem?
 
I assume so.
I just tried it out, got the out of memory, and gave up on that approach.
I guess I could bump it up to 1 gig or more, just to prove the theory but
of course at some point I'm squeamish about wasting address space or actual memory.
1gig is tiny for address space but not for actual memory.
(assuming a full 64 bit address space, which actually is probably not available by far)
 
 - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081117/2160cca5/attachment-0002.html>

From hosking at cs.purdue.edu  Mon Nov 17 19:31:01 2008
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Mon, 17 Nov 2008 12:31:01 -0600
Subject: [M3devel] gc vs. large spans of memory
In-Reply-To: <COL101-W25E6863F4A5BFE2141AA24E6130@phx.gbl>
References: <COL101-W82BC319326E633D918F12DE6100@phx.gbl>
	<COL101-W16C880C91AE709A4E8463FE6100@phx.gbl>
	<4B6BDC53-DB07-4498-A0D4-CB383DFFD487@cs.purdue.edu>
	<BAY120-W65F83A8674A6D4949CA7CE6130@phx.gbl>
	<AE6DC073-2F37-437A-8712-0C5F09B5A97D@cs.purdue.edu>
	<COL101-W25E6863F4A5BFE2141AA24E6130@phx.gbl>
Message-ID: <D67B5251-57BB-48F2-90E1-7F626B808FEA@cs.purdue.edu>

If sbrk works let's leave it at that for now.

On 17 Nov 2008, at 07:00, Jay wrote:

>  > 1 meg pages didn't suffice
>  > Is this still the sidespan array problem?
>
> I assume so.
> I just tried it out, got the out of memory, and gave up on that  
> approach.
> I guess I could bump it up to 1 gig or more, just to prove the  
> theory but
> of course at some point I'm squeamish about wasting address space or  
> actual memory.
> 1gig is tiny for address space but not for actual memory.
> (assuming a full 64 bit address space, which actually is probably  
> not available by far)
>
>  - Jay

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081117/c441cd1d/attachment-0002.html>

From rodney.bates at wichita.edu  Sun Nov 23 19:06:18 2008
From: rodney.bates at wichita.edu (Rodney M. Bates)
Date: Sun, 23 Nov 2008 12:06:18 -0600
Subject: [M3devel] The undefined ROOT problem, rebuilding cm3
In-Reply-To: <20081117103000.ld2m2zsu8w48o4ck@mail.elegosoft.com>
References: <491CB84D.4000008@wichita.edu>
	<20081117103000.ld2m2zsu8w48o4ck@mail.elegosoft.com>
Message-ID: <49299B9A.7050103@wichita.edu>

Olaf Wagner wrote:
> Hm, now I seem to be the victim of mail truncation. I'll  try a top-quote
> this time (our system administrator will investigate this). Third attempt:
> 
> It seems that this time my email wasn't complete, for whatever reasons.
> I was out of town for some days so I noticed only today.
> 
> Rodney, I did ask you if you could write up some paragraphs for the
> README in scripts and the web pages that clarifies this matter so that
> others won't run into it again?

Done.

> 
> I always seem to miss some crucial points when trying to explain the
> build system myself...
> 
> Thanks in advance,
> 
> Olaf
> 
> Quoting "Rodney M. Bates" <rodney.bates at wichita.edu>:
> 
>> Here is how the undefined ROOT problem happens when trying to rebuild
>> cm3 with itself.
>>
>> I had been making the assumption that commands build, ship, clean, when
>> supplied to the do-cm3-*.sh scripts, did the same things as -build,
>> -ship, -clean supplied to the cm3 command.  Not so.
>>
>> build actually does cm3 -build -override, as do buildlocal and empty
>> commands to the scripts. clean does cm3 -clean -override, as does 
>> cleanlocal.
>> There is a buildglobal command that does -build without -override, but
>> it also does a ship as well.  There appears to be no way to just build,
>> without either overriding or shipping.
>>
>> So, I have been running do-cm3-*.sh with no argument (and sometimes build)



From michael.anderson at elego.de  Wed Nov 26 03:10:24 2008
From: michael.anderson at elego.de (Michael Anderson)
Date: Wed, 26 Nov 2008 03:10:24 +0100
Subject: [M3devel] compiling cvsup on Debian
Message-ID: <20081126031024.o8hbvsdkgo0kw884@mail.elegosoft.com>

Howdy there,

I'm trying to compile cvsup-snap-16.1h on Debian Etch with cm3, using  
this thread as
starting point:
https://mail.elegosoft.com/pipermail/m3devel/2007-June/000468.html

Thus far, I've:

- applied the patch
- changed line 744 of FileAttr.m3 to "tz: UNTRACED REF Utime.struct_timezone;"
- changed all IMPORT directives for "SchedulerPosix" to "Scheduler",  
as well as the
corresponding procedure calls as outlined here:

https://mail.elegosoft.com/pipermail/m3devel/2008-January/001210.html

Which that leaves me with just one teensy error:

"../src/SigHandler.m3", line 221: unknown qualification '.' (IOAlertWait)

which does go away if I change the IMPORT and procedure calls back to  
SchedulerPosix for
JUST SigHandler.m3.

Is it safe to use both "Scheduler" and "SchedulerPosix", or will  
Bad_Things happen?

Thanks,
mike


-- 
Michael Anderson
IT Services & Support

elego Software Solutions GmbH
Gustav-Meyer-Allee 25
Building 12.3 (BIG) room 227
13355 Berlin, Germany

phone +49 30 23 45 86 96      michael.anderson at elegosoft.com
fax   +49 30 23 45 86 95      http://www.elegosoft.com

Geschaeftsfuehrer: Olaf Wagner, Sitz Berlin
Amtsgericht Berlin-Charlottenburg, HRB 77719, USt-IdNr: DE163214194




From hosking at cs.purdue.edu  Wed Nov 26 17:03:52 2008
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Wed, 26 Nov 2008 10:03:52 -0600
Subject: [M3devel] compiling cvsup on Debian
In-Reply-To: <20081126031024.o8hbvsdkgo0kw884@mail.elegosoft.com>
References: <20081126031024.o8hbvsdkgo0kw884@mail.elegosoft.com>
Message-ID: <192844D9-5F3B-4E15-B2E9-C540D3EBD15B@cs.purdue.edu>

Should be just fine!

On 25 Nov 2008, at 20:10, Michael Anderson wrote:

> Howdy there,
>
> I'm trying to compile cvsup-snap-16.1h on Debian Etch with cm3,  
> using this thread as
> starting point:
> https://mail.elegosoft.com/pipermail/m3devel/2007-June/000468.html
>
> Thus far, I've:
>
> - applied the patch
> - changed line 744 of FileAttr.m3 to "tz: UNTRACED REF  
> Utime.struct_timezone;"
> - changed all IMPORT directives for "SchedulerPosix" to "Scheduler",  
> as well as the
> corresponding procedure calls as outlined here:
>
> https://mail.elegosoft.com/pipermail/m3devel/2008-January/001210.html
>
> Which that leaves me with just one teensy error:
>
> "../src/SigHandler.m3", line 221: unknown qualification  
> '.' (IOAlertWait)
>
> which does go away if I change the IMPORT and procedure calls back  
> to SchedulerPosix for
> JUST SigHandler.m3.
>
> Is it safe to use both "Scheduler" and "SchedulerPosix", or will  
> Bad_Things happen?
>
> Thanks,
> mike
>
>
> -- 
> Michael Anderson
> IT Services & Support
>
> elego Software Solutions GmbH
> Gustav-Meyer-Allee 25
> Building 12.3 (BIG) room 227
> 13355 Berlin, Germany
>
> phone +49 30 23 45 86 96      michael.anderson at elegosoft.com
> fax   +49 30 23 45 86 95      http://www.elegosoft.com
>
> Geschaeftsfuehrer: Olaf Wagner, Sitz Berlin
> Amtsgericht Berlin-Charlottenburg, HRB 77719, USt-IdNr: DE163214194
>
>