Quantum simulators, machines that can replicate the dynamics of quantum systems, are being built as useful devices and are seen as a stepping stone to universal quantum computers. A key difference between the two is that computers have the ability to perform the logic gates that make up algorithms. We propose a method for learning how to construct these gates efficiently by using the simulator to perform optimal control on itself. This bypasses two major problems of purely classical approaches to the control problem: the need to have an accurate model of the system, and a classical computer more powerful than the quantum one to carry out the required simulations. Strong evidence that the scheme scales polynomially in the number of qubits, for systems of up to 9 qubits with Ising interactions, is presented from numerical simulations carried out in different topologies. This suggests that this in situ approach is a practical way of upgrading quantum simulators to computers.