You went to go see Memory Keeper just as Temple Keeper told you. He knew that you had lost your memory. To retrieve your lost memories, you need to go back to where your adventure began... Go see the one who helped you with your 1st Job Advancement.