Panama's international banking center was once a magnet for money from throughout the world, a safe haven for cash that earned the country a reputation as the Switzerland of Latin America.

Now, bankers worry the system may never recover from a nearly two-month shutdown caused by political unrest surrounding the leadership of Gen. Manuel Antonio Noriega.Frightened investors have pulled billions of dollars out of Panama. Bankers are anxiously studying their options.

I assume . . . (bankers) would be thinking about leaving the country

because they can't do much business right now, said Edgardo Lasso, president of the Banking Association of Panama, which represents 93 of the 117 mostly foreign banks operating here.

Eiichi Motoshige, general manager of the local branch of Japan's giant Sumitomo Bank, said: Each bank is very, very seriously considering what we can do for the immediate future.

The biggest problem, he said, is that nobody likes to place any funds here . . . because of the risk.

Government figures show that assets held by the banking center tumbled

from $40.3 billion at the end of 1986 to $31.4 billion at the end of 1987.

There was a massive reduction in assets beginning in June, said Luis Luis, Latin American director of the Institute of International Finance in Washington.

There must have been further shrinkage of assets in the first quarter (of this year), he said.

About three-quarters of the center's assets, or roughly $25 billion, were in off-shore activities at the end of last year.

Money in off-shore activities - those that take place outside of Panama - can be easily moved to competing banking centers in Miami, the Bahamas, Grand Cayman Island in the British West Indies and elsewhere.