Self-programming architecture

This is not a new idea -- Ben and Jared in Opencog has mentioned it before (in the context of MOSES):

Abram seems to have an idea where the GP is replaced by RL (reinforcement learning).

Yesterday I was analyzing the GP + IE idea in more details:

1. Let the GP side and the IE side gradually evolve in cycles, starting with $GP_1 + IE_1$.
2. The key question is whether the cyclic route is faster than hand-coding IE.  Initially, it would involve more work because the GP side needs to be custom-made (we cannot use off-the-shelf GP software).  It may pay off only if $GP_1 + IE_1$ increases programming productivity significantly.
3. A very weak $IE_1$ cannot increase programming productivity because GP + weak IE is still too slow to be usable.  For example, one idea is to have IE suggest a number of primitive functions when given a goal, so GP can include those primitives in the genes for that population.  But, even with current state-of-the-art GP, this cannot efficiently solve > 1 line programs, even if primitives are suggested.
4. $IE_*$ (the ideal form of IE) will be able to deduce the program when given the desired goal: $$ G:goal \mapsto \{ P:program | \quad P \vdash G \}. $$ Whereas the above $IE_1$ is too weak (suggesting primitives similar to the goal): $$ G:goal \mapsto \{ x | \quad x \approx G \}. $$ Perhaps we need to find something in between weak $IE_1$ and $IE_*$.
5. In other words, we simply have to hand-code $IE_1$ to reach a certain level of functionality before putting it to use with GP.  That basic level seems to include:
• Ability to express simple plans (so that human teachers can supply basic programming knowledge as decomposition of tasks into sub-tasks)
• Ability to express similarity and to perform simple associative recall.
  
Interestingly, the ability to perform deduction seems not required for $IE_1$, nor the ability to calculate truth values.

The new insight may change our priorities during implementation...