It's turtles all the way down
You're right about the purpose of the Fed. Interesting wikipedia article history of central banking in the US. As for where the funds came from originally, I found this description of Alexander Hamilton's plan to fund the First Bank of the United States to be a charming of bootstrapping a money supply:
According to the plan put before the first session of the First Congress, Hamilton proposed establishing the initial funding for the Bank of the United States through the sale of $10 million in stock of which the United States government would purchase the first $2 million in shares. Hamilton, foreseeing the objection that this could not be done since the U.S. government didn't have $2 million, proposed that the government make the stock purchase using money lent to it by the Bank; the loan to be paid back in ten equal annual installments. The remaining $8 million of stock would be available to the public, both in the United States and overseas. The chief requirement of these non-government purchases was that one-quarter of the purchase price had to be paid in gold or silver; the remaining balance could be paid in bonds, acceptable scrip, etc.
First Bank of the United States
That Hamilton, what a character, eh? The article goes on to calculate, somehow, that after all this boostrap-pulling, the BUS would have $500,000 in "real" money that it could lend out.
That's one example of how governments create money out of nothing. You could go back and back a long way pursuing the question of where the first money came from. But there had to be a BM and an AM, if you'll forgive the initials, and somebody had to mint the first coins (or polish the pretty rocks) to give to the stereotypical farmer in exchange for eggs.
In the beginning was an act of faith.