Environments

Orientation
1

Support for “environments” is implemented in Clojush, but the utility of environments has not yet been extensively tested. It is possible that the feature will be refined after such testing takes place.

Here is @thelmuth’s brief description of environments, as currently implemented, copied from a github issue thread:

The idea behind environments is to execute some code in a local scope, so that the code doesn’t affect the other stacks.

There are two ways to start an environment. environment_new takes the top block of code on the :exec stack and executes it in a new environment. The environment_begin instruction does the same thing, but uses the entire rest of the :exec stack. Environments can be ended in two ways: running out of code on the :exec stack, and reaching an environment_end instruction.

When an environment starts, it pushes the current Push state onto the :environment stack. When it finishes, it pops the top of the environment stack, restoring the state as it was before the environment started. The only way for code inside an environment to affect the rest of the program is to use return_X and return_Y_pop instructions. The return_X instructions have the name of a stack instead of X. When such an instruction is executed, that type is put onto the :return stack, and when the environment finishes it will put the top item on the X stack onto the X stack in the base environment. The return_Y_pop instructions are similar, except they specify a stack to pop after returning from the environment, allowing a program to “consume” arguments.

1 Like
2

Can I assume (haven’t checked the code) that the counter is copied from the current interpreter to the new interpreter, so the termination conditions do not get reset?

3

This is correct.

In fact, there isn’t a new interpreter at all, but instead, the old push-state is pushed onto the :environments stack in the new environment. When the environment finishes, the old state is popped off the environment stack and restored.

5

There are 2 ways an environment can start:

  1. environment_new, which creates a new environment with the first thing on the old :exec stack as the only thing on the new :exec stack.
  2. environment_begin, which creates a new environment with everything that was on the old :exec stack on the new :exec stack.

There are two corresponding ways to end an environment. Note that while the first way goes along well with environment_new and the second way goes well with environment_begin, either ending method will end an environment, no matter what instruction created it.

  1. The :exec stack is empty. If this happens and there is something on the :environment stack, then we have come to the end of an environment, and it’s time to pop it. (Note that this means a program will never terminate normally with something on the :environment stack).
  2. environment_end, which ends the current environment immediately, and puts everything on the :exec stack in the environment at the head of the :exec stack in the return environment. This is because this instruction is “meant” to be paired with environment_begin, and without putting stuff on the :exec stack, you would never be able to use these instructions and have something happen after them.

With all of this in mind, with the first environment ending scenario above, nothing will be on the :exec stack anyway, so the end-environment function will concat an empty old :exec onto the new :exec. In the second environment ending scenario, we want to have the code from the :exec stack transferred to the returning environment.

1 Like
6

So this really ends up being an amendment to the original “program execution is done” check, doesn’t it?

7

Yup. Hence this line in the Clojush interpreter, which says to terminate if both the :exec stack and :environment stack are empty.

8

I think if it’s OK with everybody, I’m going to do the same, and :environments (and :return behavior) will be cooked into the core interpreter functionality, rather than an add-on.

1 Like
9

One more clarification: the resulting :exec stack is a concatenation of three sources, right? The contents of the sub-environment’s :result stack, the contents of the sub-environment’s :exec stack, and the contents of the stored environment’s :exec stack, in that order?

:exec '(environment_end 1 1 111)
:result '(4 44 444)
:env  '( {:exec '(9 99 999) } )

;; becomes

:exec '(4 44 444 1 1 111 9 99 999)
:result '( )
:env  '( )

Right? I know this isn’t the “designed pairing”, but I’m testing the undesigned consequences.

10

I don’t know what a :result stack is. Do you mean :return?

If so, pretty good, but I think you have the things coming off the :return stack backwards, since they should be concated in reverse order:

(execute-instruction 'environment_end
                        (assoc (make-push-state)
                               :exec '(1 2 3)
                               :return '(4 5 6)
                               :environment '({:exec (7 8 9) :integer '(1111 2222 3333)})
                               :integer '(100 200 300)
                               ))
{:exec (6 5 4 1 2 3 7 8 9),
 :integer '(1111 2222 3333),
 :auxiliary nil}
11

Great! That clarifies.

But the same thing (exactly) happens for a “soft” environment ending, right? Although the :exec stack will be empty, the :return stack will still be pushed onto the retrieved “old” :exec?

12

This is correct.

13

Looking into doing more experiments with environments, particularly in the context of autoconstruction, I realized that we don’t currently have return instructions for genomes. And looking into adding those, I realize that this trick of doing returns via the :exec stack fails for types without a syntax for literals.

It seems to me that the best approach would be to change all of the return instructions so that they don’t just push raw values onto the :return stack, but instead push type/value pairs. Then, instead of pushing everything from the :return stack onto the :exec stack when we leave an environment, push everything right onto the appropriate destination stacks.

Does that seem reasonable?

14

This seems entirely reasonable to me. This would also make return_fromcode more inline with the other instructions.

The only functional difference I see between that and our current code is that as is, it can intermix returned literals and code to the exec stack, which means returned code to the exec stack could manipulate other returned values before they are pushed onto their typed stacks. But, that is a functionality I am fine giving up for this circumstance.

One alternative, which I’m only about 60% sure would work, would be to use metadata to mark the genomes (which are lists?) on the exec stack, and then have a function in literals that recognizes genomes based on this metadata. But, this sounds hacky and maybe a bad use of metadata?

2 Likes
15

Genomes are vectors… and using meta-data for something like this makes me a little squeamish because it’s so easy to lose it without noticing… So I think I’ll go with the first approach when I get a chance.

2 Likes
16

Finally focused on this and think that the possibility you describe is pretty cool, because the code you return can sort of meta-program the way values are returned. However, it also seems to me that it could do essentially the same thing anyway, post-processing the values from the typed stacks rather than from the :exec stack… so I (also) don’t see this as a reason not to make return instructions send things straight to the typed stacks.

2 Likes
17

BTW, if you’re making these changes, I wouldn’t be opposed to moving the environments instructions from code.clj to a new environments_and_return.clj file. I don’t remember why I put them with the code instructions in the first place. :confused:

2 Likes
18

I did this, but named the file (and namespace) just environment.

This is currently just in the auto-instruction-sets branch.

@thelmuth (especially, since he implemented environments, but anyone else should also feel free to do this), I’d appreciate it if you could sanity check what I’ve done here. There are a few other changes from master in this branch, which I made before the changes to environments, but they’re minor and distinct, and I made pretty fine-grained commits/messages so it should be relatively easy to see just the environment-related changes (including some earlier screw-ups, so definitely start from the final commit).

2 Likes
19

Just reading the code, it looks like it should work. Assuming you have tested it and it gave your expected results, I’d say you’re good to go!

That said, I’m realizing that the behavior here, which I think is the same as the original, is that if a return_pop_integer instruction is followed by a return_integer instruction (or maybe the other way around?), they will cancel each other out by pushing and then popping, instead of popping something in the original environment and then pushing. It seems like it would be nice to always have return_pop instructions only pop things in the environment you’re returning to, instead of things that are being returned, but maybe there are reasons against this. To reiterate, I think your code does exactly what my old code did; I’m questioning whether this is actually wise.

1 Like
20

A phrase that should probably be spoken more frequently by programmers… :stuck_out_tongue_winking_eye:

2 Likes
21

I’ve merged the changes described above into master and pushed it to github.

I’ll include a few tests below that show how a couple of things (still) work, including one that confirms @thelmuth’s comment above that it’s possible to pop the value that you’re trying to return from an environment, rather than the thing in the parent environment that you might want to replace, if you do things in the wrong order. Not sure if that’s a bug or a feature, but it is what it is at least for now.

-Lee

Here we return a 2 from an environment and add it to a 1:

(run-push '(1 environment_begin 2 return_frominteger environment_end integer_add) 
          (make-push-state))

{:exec (), :code nil, :integer (3), :float nil, :boolean nil, :char nil, :string nil, :zip nil, :vector_integer nil, :vector_float nil, :vector_boolean nil, :vector_string nil, :input nil, :output nil, :auxiliary nil, :tag nil, :return nil, :environment nil, :genome nil, :termination :normal}

Here we fail to return the 2, so the add no-ops and we’re left with just the 1:

(run-push '(1 environment_begin 2 environment_end integer_add) 
          (make-push-state))

{:exec (), :code nil, :integer (1), :float nil, :boolean nil, :char nil, :string nil, :zip nil, :vector_integer nil, :vector_float nil, :vector_boolean nil, :vector_string nil, :input nil, :output nil, :auxiliary nil, :tag nil, :return nil, :environment nil, :genome nil, :termination :normal}

Here we return a 10 from the environment and pop as we do so, so the 10 replaces the 1 on top of the :integer stack and the 10 gets added to the 100 that was below it:

(run-push '(100 1 
                environment_begin 10 return_frominteger return_integer_pop environment_end 
                integer_add) 
          (make-push-state))

{:exec (), :code nil, :integer (110), :float nil, :boolean nil, :char nil, :string nil, :zip nil, :vector_integer nil, :vector_float nil, :vector_boolean nil, :vector_string nil, :input nil, :output nil, :auxiliary nil, :tag nil, :return nil, :environment nil, :genome nil, :termination :normal}

Here we do the pop too early (yeah, it’s counterintuitive that the problem is earliness – thank the first-in-last-out nature of stacks for that) and the thing that gets popped is the 10 that we were also returning, so the add gets applied to the 100 and the 1:

(run-push '(100 1 
                environment_begin return_integer_pop 10 return_frominteger environment_end 
                integer_add) 
          (make-push-state))

{:exec (), :code nil, :integer (101), :float nil, :boolean nil, :char nil, :string nil, :zip nil, :vector_integer nil, :vector_float nil, :vector_boolean nil, :vector_string nil, :input nil, :output nil, :auxiliary nil, :tag nil, :return nil, :environment nil, :genome nil, :termination :normal}
2 Likes