Edging Armageddon – by Peruvian Bull

CategoriesGamestop_, Issue 2023Q2
The Republicans and Democrats are using the debt limit to again play nuclear brinkmanship. The deal they’ve reached is now more terrifying than I could ever have imagined.

From: https://peruvianbull.substack.com/p/edging-armageddon

This past weekend, we were subject to another close shave with Treasury default. Speaker Kevin McCarthy and the Biden Administration were the newest contenders in a fight that is becoming all too common in our current fiscal landscape: the debt ceiling.

The debt ceiling, also known as the statutory debt limit, is a legal cap on the amount of debt the United States government can accumulate to finance its spending obligations. It is set by Congress, and any increase requires their approval. Essentially, the debt ceiling acts as a self-imposed limit to control the government’s borrowing capacity.

The current form of the debt ceiling took shape with the passage of the Second Liberty Bond Act in 1917, during World War I. This act established limits on the total amount of government debt that could be outstanding at any given time. It marked a significant shift in Congress’s approach, establishing a more fixed and permanent debt limit.

Over the years, several pieces of legislation have shaped the debt ceiling’s operation. The Public Debt Acts of 1939 and 1941 introduced the concept of a statutory debt limit, explicitly setting a dollar amount for government borrowing. The Graham-Rudman-Hollings Balanced Budget and Emergency Deficit Control Act of 1985 attempted to link the debt ceiling with deficit reduction targets, imposing automatic spending cuts if those targets were not met.

The primary objective of the debt ceiling is to ensure that the government does not overspend or accumulate an unsustainable amount of debt. It serves as a “mechanism” to encourage fiscal responsibility and maintain the credibility of the United States.

However, this limit has long since lost any and all meaning. Per the Treasury’s own website

“Since 1960, Congress has acted 78 separate times to permanently raise, temporarily extend, or revise the definition of the debt limit – 49 times under Republican presidents and 29 times under Democratic presidents.”

The ceiling has been raised by both parties, under virtually every President that has held the throne of power for the last 60 years.

Once a tool of financial restraint, it has been morphed into a political hot-potato- a game of political chicken where the parties use the impending doom of default to push through last minute spending and political priorities.

They’re using essentially nuclear brinkmanship to get things done.

The irresponsibility and shortsightedness of this cannot be overstated- as a true Treasury default would be catastrophic for the global financial system. I laid out so much in a Twitter thread on Friday, which I will lay out the basics here for review:

If the government hits the debt ceiling and Congress fails to raise it, the Treasury Department must resort to extraordinary measures to continue meeting its financial obligations. These measures include reallocating funds from various government accounts, suspending the issuance of certain types of debt, and implementing cash management techniques. However, these measures are temporary solutions and can only buy the government a limited amount of time before it runs out of funds.

In the case of the United States, this would mean the government not being able to pay interest or principal on its Treasury bonds, bills, and notes held by investors.

The first and most immediate consequence of a U.S. debt default would be a loss of confidence in American financial stability.

Treasury bonds have long been considered a safe haven investment, and a default would cause ripple effects in the many markets that use Treasuries as collateral- the repo markets, FX, swap, and money markets being just a few examples.

The value of the U.S. dollar would likely plummet as investors lose faith in its stability. This would lead to a decrease in the purchasing power of the currency, making imported goods more expensive and fueling inflation.

The EM crises that hit Thailand, Argentina and others would now come to roost in the States.

This is a serious problem because unlike other countries, the US does not have sufficient foreign exchange reserves as it is the world reserve currency.

Other countries have trillions of dollars in the vaults. We have $38B.

Interest rates would spike as investors demand higher returns to compensate for the increased risk associated with U.S. debt. This would affect borrowing costs for businesses and consumers, making it more expensive to obtain credit for investments, mortgages, and other loans.

This is occurring at record Federal debt levels- further pushing the US into a debt spiral.

Financial markets would experience significant turmoil. Stock markets could plummet as investor confidence wavers, leading to widespread panic selling.

Pension funds, mutual funds, and other investment vehicles heavily exposed to U.S. government debt would suffer substantial losses, similar to how the UK’s pensions blew up as their leveraged UK bond positions were all liquidated-


The U.S. government would face challenges in raising funds to finance its operations. Without access to borrowing through Treasury securities, the government would need to rely solely on tax revenues and other limited sources of income. This could result in significant cuts to government programs and services- and government spending is a component of GDP.

We would see a collapse of GDP of several percent in just a month or two- reminiscent of the COVID shutdowns.

The global economy is intricately interconnected, and a default would disrupt markets worldwide, triggering a global financial crisis similar to the impact of the Lehman Brothers collapse in 2008.

Recall there is over $7T of Treasuries held by foreigners globally- if the US defaults, and it is not remedied immediately, then theoretically these sovereigns could begin dumping their huge Treasury debt positions on the market.

Credit rating agencies would likely downgrade the United States’ sovereign debt rating. This would further increase borrowing costs, making it even harder for the government and businesses to raise capital.

In fact, this is already beginning to happen-


The consequences of a U.S. debt default would also have long-lasting effects on the country’s reputation as a reliable borrower. It could take years, if not decades, to rebuild trust among investors and regain the status of a safe haven for global capital.


Default is an unacceptable outcome, and barring an increase in the limit, the Treasury would resort to austerity measures to ensure coupon payments continue to flow out- however with deficit spending still raging, this means severe cuts to many government programs. Social Security, pensions, and veteran’s benefits would be first on the chopping block- but many more would follow in time.

Thus our worst case scenario of a “technical” default would be averted- but at what cost? Recall Treasury would have to issue new bills to pay off maturing ones, a scheme I have described eerily reminiscent of a Ponzi scheme…

And all of this at record high interest rates! The yield on the new bills would be 5.25%, pushing the Treasury deeper into a debt spiral in even the best case scenario. Their auction schedule for the next few weeks can be seen below:

However, we can all breathe a giant sigh of relief. A new debt limit deal was reached Saturday afternoon, and unsurprisingly to those of us who follow this monetary mayhem, it promises to do nothing but make the situation worse. Congress still has to pass this deal, of course.

Here are the key aspects of the deal, provided by the NYT:

  • Complete suspension of debt limit for 2 years, until 2025
  • New work requirements for certain recipients of food stamps and the Temporary Aid for Needy Families program.
  • It would limit all discretionary spending to 1 percent growth in 2025
  • accelerate the permitting of some energy projects- such as a  new natural gas pipeline from West Virginia to Virginia.
  • The debt limit agreement would immediately rescind $1.38 billion from the I.R.S. and ultimately repurpose another $20 billion from the $80 billion it received through the Inflation Reduction Act.
  • The proposed military spending budget would increase to $886 billion next year, which is in line with what Mr. Biden requested in his 2024 budget proposal, and rise to $895 billion in 2025.
  • New York Times analysis of the proposal suggests it would reduce federal spending by about $55 billion next year, compared with Congressional Budget Office forecasts, and by another $81 billion in 2025. (Drops in the bucket considering the trillions of dollars of spending the government disburses annually)

Perhaps most importantly, this section was pointed out by ZeroHedge:

Although Republicans had initially called for 10 years of spending caps, this legislation includes just 2 years of caps and then switches to spending targets that are not bound by law — essentially, just suggestions.

Essentially what this means, as one Twitter user pointed out, is a removal of the budgetary limits on most government spending programs. Historically how government agencies operate, is they are given a budget and allowed to spend an amount, up to a cap, for a fiscal year. If they run close to that limit, the next year they are given the same limit, and usually plus a few extra percent, to achieve their priorities through that next calendar year.

After 2025, there will be no limits- the agencies will spend HOWEVER much they deem they need. They’ll worry about the bill later.

This is the stuff of banana republics…

Can you smell that?

Time to get your printer ready, Jerome.

A deep dive into the shrinking money supply: Banks get sweetheart programs from the liquidity fairy while households are forced to take on debt and in some instances DIE while being priced out of their lives in favor of rising interest rates to fight an inflation problem the Fed created.

CategoriesGamestop_, Gamestop., Issue 2023Q2

by u/ Dismal-Jellyfish

Good morning and Happy Wednesday Superstonk! Before I get started, fun fact, did you know a group of jellyfish is called a smack?

With that, I hope y’all will join me as we ‘smack this fish up‘ while we dive into today’s topic: Shrinking M2.

I would like to take a minute to review some of the data around the shrinking money stock, the borrowing banks are able to take utilize vs the debt households are taking on.

I hope by the end of this post, it will be clear that Banks and Households are not experiencing this current economic environment the same way.

In my opinion, inflation is the big bad boogey man that has kicked off the need for all of this borrowing.

While this post is not about inflation specifically, it is fascinating to watch inflation continue to rage even as money stock continues to drop.

Let’s get to it!

M2 (U.S. money stock–currency and coins held by the non-bank public, checkable deposits, and travelers’ checks, plus savings deposits, small time deposits under 100k, and shares in retail money market funds) is decreasing:

  1. M1 consists of (1) currency outside the U.S. Treasury, Federal Reserve Banks, and the vaults of depository institutions; (2) demand deposits at commercial banks (excluding those amounts held by depository institutions, the U.S. government, and foreign banks and official institutions) less cash items in the process of collection and Federal Reserve float; and (3) other liquid deposits, consisting of other checkable deposits (or OCDs, which comprise negotiable order of withdrawal, or NOW, and automatic transfer service, or ATS, accounts at depository institutions, share draft accounts at credit unions, and demand deposits at thrift institutions) and savings deposits (including money market deposit accounts). Seasonally adjusted M1 is constructed by summing currency, demand deposits, and other liquid deposits, each seasonally adjusted separately.
  2. M2 consists of M1 plus (1) small-denomination time deposits (time deposits in amounts of less than $100,000) less individual retirement account (IRA) and Keogh balances at depository institutions; and (2) balances in retail money market funds (MMFs) less IRA and Keogh balances at MMFs. Seasonally adjusted M2 is constructed by summing small-denomination time deposits and retail MMFs, each seasonally adjusted separately, and adding the result to seasonally adjusted M1.
  3. Currency in circulation consists of Federal Reserve notes and coin outside the U.S. Treasury and Federal Reserve Banks.
  4. Reserve balances are balances held by depository institutions in master accounts and excess balance accounts at Federal Reserve Banks.
  5. Monetary base equals currency in circulation plus reserve balances.
  6. Total reserves equal reserve balances plus, before April 2020, vault cash used to satisfy reserve requirements.
  7. Total borrowings in millions of dollars from the Federal Reserve are borrowings from the discount window’s primary, secondary, and seasonal credit programs and other borrowings from emergency lending facilities. For borrowings included, see “Loans” in table 1 of the H.4.1 statistical release.
  8. Nonborrowed reserves equal total reserves less total borrowings from the Federal Reserve.

A little less than a year ago (July 2022) the M2 high was hit at $21,703 billion

Date M2 (billions) Down from all time high (billions)
July 2022 $21,703 0
August 2022 $21,660 -$43 billion
September 2022 $21,524 -$179 billion
October 2022 $21,432 -$271 billion
November 2022 $21,398 -$305 billion
December 2022 $21,358 -$345 billion
January 2023 $21,212 -$491 billion
February 2023* $21,076 -$627 billion
March 2023 $20,840 -$863 billion
April 2023 $20,673 -$1030 billion

*Bank run in commercial banks picked up in February 2022.

While M2 is dropping, deposits at all Commercial Banks are Shrinking:

Source: https://www.federalreserve.gov/releases/h8/20230519/

Domestically chartered commercial banks divested $87 billion in assets to nonbank institutions in the week ending March 29, 2023. The major asset item affected was the following: securities, $87 billion.

Domestically chartered commercial banks divested $87 billion in assets to nonbank institutions in the week ending March 22, 2023. The major asset items affected were the following: securities, $27 billion; and loans, $60 billion.

A little over a year ago (4/13/2022) the high was hit at $18,158.3536 billion

Date Deposits, All Commercial Banks (billions) Down from all time high (billions)
4/13/2022 $18,158 0
2/22/2023 (Run picks up speed) $17,690 -$468 billion
3/1/2023 $17,662 -$496 billion
3/8/2023 $17,599 -$559 billion
3/15/2023 $17,428 -$730 billion
3/22/2023 $17,256 -$902 billion
3/29/2023 $17,192 -$966 billion
4/5/2023 $17,253 -$905 billion
4/12/2023 $17,168 -$990 billion
4/19/2023 $17,180 -$978 billion
4/26/2023* $17,164 -$994 billion
5/3/2023 $17,149 -$1,009 billion
5/10/2023 $17,123 -$1,035 billion

*April is the most up to date M2 numbers

However, borrowing from the liquidity fairy is spiraling to make up for it!:

Bank Term Funding Program (BTFP)

Tool Bank Term Funding Program (BTFP) Up from 3/15, 1st week of program ($ billion)
3/15) $11.943 billion $0 billion
3/22 $53.669 billion $41.723 billion
3/29 $64.403 billion $52.460 billion
3/31 $64.595 billion $52.652 billion
4/5 $79.021 billion $67.258 billion
4/12 $71.837 billion $59.894 billion
4/19 $73.982 billion $62.039 billion
4/26 $81.327 billion $69.384 billion
5/3 $75,778 billion $63.935 billion
5/10 $83,101 billion $71.158 billion
5/17 $87,006 billion $75.063 billion


r/Superstonk - https://www.reddit.com/r/Superstonk/comments/11prthd/federal_reserve_alert_federal_reserve_board/
  • Association, or credit union) or U.S. branch or agency of a foreign bank that is eligible for primary credit (see 12 CFR 201.4(a)) is eligible to borrow under the Program.
  • Banks can borrow for up to one year, at a fixed rate for the term, pegged to the one-year overnight index swap rate plus 10 basis points.
  • Banks have to post collateral (valued at par!).
  • Any collateral has to be “owned by the borrower as of March 12, 2023.”
  • Eligible collateral includes any collateral eligible for purchase by the Federal Reserve Banks in open market operations.

“Other credit extensions”

Tool Other Credit Extension Up from 3/15, 1st week of program ($ billion)
3/15) $142.8 billion $0 billion
3/22 $179.8 billion $37 billion
3/29 $180.1 billion $37.3 billion
4/5 $174.6 billion $31.8 billion
4/12 $172.6 billion $29.8 billion
4/19 $172.6 billion $29.8 billion
4/26 $170.3 billion $27.5 billion
5/3 $228.2 billion $85.4 billion
5/10 $212.5 billion $69.7billion
5/17 $208.5 billion $65.7 billion

“Other credit extensions” includes loans that were extended to depository institutions established by the Federal Deposit Insurance Corporation (FDIC). The Federal Reserve Banks’ loans to these depository institutions are secured by collateral and the FDIC provides repayment guarantees.

For example, $114 billion in face value Agency Mortgage Backed Securities, Collateralized Mortgage Obligations, and Commercial Mortgage Backed Securities about to be liquidated ‘gradual and orderly’ with the ‘aim to minimize the potential for any adverse impact on market functioning’ by BlackRock.

How I understand this works:

  • The FDIC created temporary banks to support the operations of the ones they have taken over.
  • The FDIC did not have the money to operate these banks.
  • The Fed is providing that in the form of a loan via “Other credit extensions”.
  • The FDIC is going to sell the taken over banks assets.
  • Whatever the difference between the sale of the assets and the ultimate loan number is, will be the amount split up amongst all the remaining banks and applied as a special fee to make the Fed ‘whole’.
  • It can be argued the consumer will ultimately end up paying for this as banks look to pass this cost on in some way.

There has been an update on this piece recently:

Whatever the difference between the sale of the assets and the ultimate loan number is, will be the amount split up amongst all the remaining banks and applied as a special fee to make the Fed ‘whole’.

FDIC Board of Directors Issues a Proposed Rule on Special Assessment Pursuant to Systemic Risk Determination of approximately $15.8 billion. It is estimated that a total of 113 banking organizations would be subject to the special assessment.


What does all this borrowing look like for the banks?


Over the few weeks prior to the FDIC receivership announcements on March 10 and 12, the banking sector lost another approximately $450 billion. Throughout, the banking sector has offset the reduction in deposit funding with an increase in other forms of borrowing which has increased by $800 billion since the start of the tightening.

The right panel of the chart below summarizes the cumulative change in deposit funding by bank size category since the start of the tightening cycle through early March 2023 and then through the end of March. Until early March 2023, the decline in deposit funding lined up with bank size, consistent with the concentration of deposits in larger banks. Small banks lost no deposit funding prior to the events of late March. In terms of percentage decline, the outflows were roughly equal for regional, super-regional, and large banks at around 4 percent of total deposit funding:

The blue bar in the left panel above shows that the pattern changes following the run on SVB. The additional outflow is entirely concentrated in the segment of super-regional banks. In fact, most other size categories experience deposit inflows.

The right panel illustrates that outflows at super-regionals begin immediately after the failure of SVB and are mirrored by deposit inflows at large banks in the second week of March 2022.

Further, while deposit funding remains at a lower level throughout March for super-regional banks, the initially large inflows mostly reverse by the end of March. Notably, banks with less than $100 billion in assets were relatively unaffected.

However, during the most acute phase of banking stress in mid-March, other borrowings exceeded reductions in deposit balances, suggesting significant and widespread demand for precautionary liquidity. A substantial amount of liquidity was provided by the private markets, likely via the FHLB system, but primary credit and the Bank Term Funding Program (both summarized as Federal Reserve credit) were equally important.

  • Large banks increased borrowing the most, which is in line with deposit outflows being strongest for larger banks before March 2023.
  • During March 2023, both super-regional and large banks increase their borrowings, with most increases being centered in the super-regional banks that faced the largest deposit outflows.
  • Note, however, that not all size categories face deposit outflows but that all except the small banks increase their other borrowings.
  • This pattern suggests demand for precautionary liquidity buffers across the banking system, not just among the most affected institutions:
  • So, on the commercial side, as M2 has been shrinking, the banking system has seen a considerable decline in deposit funding since the start of the current monetary policy tightening cycle in March 2022.
  • The speed of deposit outflows increased during March 2023, following the run on SVB, with the most acute outflows concentrated in a relatively narrow segment of the banking system, super-regional banks (those with $50 to $250 billion in total assets).
  • Notably, deposit funding amongst the cohort often referred to as community and smaller regional banks (that is, institutions with less than $50 billion in assets) were relatively stable by comparison.
  • Large banks (those with more than $250 billion in assets), which had been subject to the largest deposit outflows before March 2023, received deposit inflows throughout March 2023.
  • Throughout, banks were able to replace deposit outflows by making use of alternative funding sources–FHLB, Primary Credit, BTFP.

Banks get liquidity while ‘we’ get inflations and rate hikes. Speaking of households…

During this same period, Household borrowing has also skyrocketed!

From 1st quarter 2022 to 1st quarter 2023, total household debt has increased $1,205 billion to $17.05 trillion (+7.57%)–Mortgage balances ($864 billion), HELOC ($22 billion), Student loans ($14 billion), Auto loans ($93 billion), Credit Card debt ($145 billion), Other ($67 billion):

  • Total household debt has risen by $148 billion, or 0.9 percent, to $17.05 trillion in the first quarter of 2023.
  • Mortgage balances climbed by $121 billion and stood at $12.04 trillion at the end of March.
  • Auto loans to $1.56 trillion.
  • Student loans to $1.60 trillion.
  • Credit Card debt $986 billion.

However, unlike the banks above, there are no fancy programs designed to keep households afloat in this inflating economy–and boy are households starting to feel it, especially in the areas like services and housing (that are BIG components of CPI–and way more ‘sticky’ than goods).

For example, on the housing front:

April 2023 Rental Report: The median asking rent was $1,734, up by $4 from last month and down by $43 from the peak but still $348 (25.1%) higher than the same time in 2019 (pre-pandemic)

To try and further drive home the shaky ground households are on, let’s revisit the Fed’s Economic Well-being US Household 2022.

  • “fewer adults reported having money left over after paying their expenses. 54% of adults said that their budgets had been affected “a lot” by price increases.”
  • “51% of adults reported that they reduced their savings in response to higher prices.”
  • The share of adults who reported that they would cover a $400 emergency expense using cash or its equivalent was 63 percent.

It is the younger generations starting to see itself break into delinquency now:

  • Auto loans are above 3% delinquency for (30-39) and approaching 5% for (18-29)
  • Credit Cards are above 6% delinquency for (30-39) and approaching 9% for (18-29)
  • Student Loan delinquency is being artificially suppressed currently.
    • Speculation: when folks (18-29) and (30-39) have to pay Auto loans, Credit Card dent, and Student loans all at the same time, delinquencies across all 3 will jump bigly.
    • People will DIE being priced out of their lives in favor of raising interest rates to fight inflation for a problem the Fed created to begin with:


  • M2 is shrinking
  • Borrowing is up
    • Banks have access to sweetheart programs from the liquidity fairy.
    • Households are taking on debt that is literally killing them.
  • Some fed governors are calling for 2 more rate hikes this year
r/Superstonk - A deep dive into the shrinking money supply: Banks get sweetheart programs from the liquidity fairy while households are forced to take on debt and in some instances DIE while being priced out of their lives in favor of rising interest rates to fight an inflation problem the Fed…

Introducing MPT-7B: A New Standard for Open-Source, Commercially Usable LLMs

CategoriesAI-ML_, Issue 2023Q2, Site Updates_

From: https://www.mosaicml.com/blog/mpt-7b

Introducing MPT-7B, the latest entry in our MosaicML Foundation Series. MPT-7B is a transformer trained from scratch on 1T tokens of text and code. It is open source, available for commercial use, and matches the quality of LLaMA-7B. MPT-7B was trained on the MosaicML platform in 9.5 days with zero human intervention at a cost of ~$200k. Starting today, you can train, finetune, and deploy your own private MPT models, either starting from one of our checkpoints or training from scratch. For inspiration, we are also releasing three finetuned models in addition to the base MPT-7B: MPT-7B-Instruct, MPT-7B-Chat, and MPT-7B-StoryWriter-65k+, the last of which uses a context length of 65k tokens!

U.S. officials at the federal and state level are assessing the possibility of “market manipulation” behind big moves in banking share prices in recent days by Short Sellers

CategoriesGamestop_, Issue 2023Q2

Credit goes to Reuters :

May 4 (Reuters) – U.S. officials at the federal and state level are assessing the possibility of “market manipulation” behind big moves in banking share prices in recent days, a source familiar with the matter said on Thursday.

Shares of regional banks resumed their slide this week after the collapse of First Republic Bank, the third U.S. mid-sized lender to fail in two months. Short sellers raked in $378.9 million in paper profits on Thursday alone from betting against certain regional banks, according to analytics firm Ortex.

Increased short-selling activity and volatility in shares have drawn increasing scrutiny by federal and state officials and regulators in recent days, given strong fundamentals in the sector and sufficient capital levels, said the source, who was not authorized to speak publicly.

“State and federal regulators and officials are increasingly attentive to the possibility of market manipulation regarding banking equities,” the source said.

PacWest Bancorp (PACW.O) shares slid 57% on Thursday, dragging down other regional lenders, after the Los Angeles-based bank said it was in talks about strategic options.

Western Alliance Bancorp (WAL.N) denied a report from the Financial Times that said it was exploring a potential sale, and said it was exploring legal options. The report had sent the lender’s shares down as much as 61.5% before trading was halted.

Share price swings did not reflect the fact that many regional banks outperformed on first quarter earnings and had sound fundamentals, including stable deposits, sufficient capital, and decreased uninsured deposits, the source said.

“This week we have seen that regional banks remain well- capitalized,” the source said.

Short selling, in which investors sell borrowed securities and aim to buy these back at a lower price to pocket the difference, is not illegal and considered part of a healthy market. But manipulating stock prices, which the SEC has defined as the ‘intentional or willful conduct designed to deceive or defraud investors by controlling or artificially affecting” stock prices, is.

An official with the U.S. Securities and Exchange Commission told Reuters on Wednesday the agency was “not currently contemplating” a short-selling ban.

On Thursday the agency did not respond immediately to a Reuters request for comment.

But the source familiar with current events noted that the agency had warned in March, during a previous period of high market volatility surrounding the collapse of Silicon Valley Bank and Signature Bank, that it was carefully monitoring market stability and would prosecute any form of misconduct.

Google “We Have No Moat, And Neither Does OpenAI”

CategoriesIssue 2023Q2, Site Updates_
Leaked Internal Google Document Claims Open Source AI Will Outcompete Google and OpenAI

Please consider supporting the source: https://www.semianalysis.com/p/google-we-have-no-moat-and-neither

We’ve done a lot of looking over our shoulders at OpenAI. Who will cross the next milestone? What will the next move be?

But the uncomfortable truth is, we aren’t positioned to win this arms race and neither is OpenAI. While we’ve been squabbling, a third faction has been quietly eating our lunch.

I’m talking, of course, about open source. Plainly put, they are lapping us. Things we consider “major open problems” are solved and in people’s hands today. Just to name a few:

While our models still hold a slight edge in terms of quality, the gap is closing astonishingly quickly. Open-source models are faster, more customizable, more private, and pound-for-pound more capable. They are doing things with $100 and 13B params that we struggle with at $10M and 540B. And they are doing so in weeks, not months. This has profound implications for us:

  • We have no secret sauce. Our best hope is to learn from and collaborate with what others are doing outside Google. We should prioritize enabling 3P integrations.
  • People will not pay for a restricted model when free, unrestricted alternatives are comparable in quality. We should consider where our value add really is.
  • Giant models are slowing us down. In the long run, the best models are the ones

    which can be iterated upon quickly. We should make small variants more than an afterthought, now that we know what is possible in the <20B parameter regime.


What Happened

At the beginning of March the open source community got their hands on their first really capable foundation model, as Meta’s LLaMA was leaked to the public. It had no instruction or conversation tuning, and no RLHF. Nonetheless, the community immediately understood the significance of what they had been given.

A tremendous outpouring of innovation followed, with just days between major developments (see The Timeline for the full breakdown). Here we are, barely a month later, and there are variants with instruction tuningquantizationquality improvementshuman evalsmultimodalityRLHF, etc. etc. many of which build on each other.

Most importantly, they have solved the scaling problem to the extent that anyone can tinker. Many of the new ideas are from ordinary people. The barrier to entry for training and experimentation has dropped from the total output of a major research organization to one person, an evening, and a beefy laptop.

Why We Could Have Seen It Coming

In many ways, this shouldn’t be a surprise to anyone. The current renaissance in open source LLMs comes hot on the heels of a renaissance in image generation. The similarities are not lost on the community, with many calling this the “Stable Diffusion moment” for LLMs.

In both cases, low-cost public involvement was enabled by a vastly cheaper mechanism for fine tuning called low rank adaptation, or LoRA, combined with a significant breakthrough in scale (latent diffusion for image synthesis, Chinchilla for LLMs). In both cases, access to a sufficiently high-quality model kicked off a flurry of ideas and iteration from individuals and institutions around the world. In both cases, this quickly outpaced the large players.

These contributions were pivotal in the image generation space, setting Stable Diffusion on a different path from Dall-E. Having an open model led to product integrationsmarketplacesuser interfaces, and innovations that didn’t happen for Dall-E.

The effect was palpable: rapid domination in terms of cultural impact vs the OpenAI solution, which became increasingly irrelevant. Whether the same thing will happen for LLMs remains to be seen, but the broad structural elements are the same.

What We Missed

The innovations that powered open source’s recent successes directly solve problems we’re still struggling with. Paying more attention to their work could help us to avoid reinventing the wheel.

LoRA is an incredibly powerful technique we should probably be paying more attention to

LoRA works by representing model updates as low-rank factorizations, which reduces the size of the update matrices by a factor of up to several thousand. This allows model fine-tuning at a fraction of the cost and time. Being able to personalize a language model in a few hours on consumer hardware is a big deal, particularly for aspirations that involve incorporating new and diverse knowledge in near real-time. The fact that this technology exists is underexploited inside Google, even though it directly impacts some of our most ambitious projects.

Retraining models from scratch is the hard path

Part of what makes LoRA so effective is that – like other forms of fine-tuning – it’s stackable. Improvements like instruction tuning can be applied and then leveraged as other contributors add on dialogue, or reasoning, or tool use. While the individual fine tunings are low rank, their sum need not be, allowing full-rank updates to the model to accumulate over time.

This means that as new and better datasets and tasks become available, the model can be cheaply kept up to date, without ever having to pay the cost of a full run.

By contrast, training giant models from scratch not only throws away the pretraining, but also any iterative improvements that have been made on top. In the open source world, it doesn’t take long before these improvements dominate, making a full retrain extremely costly.

We should be thoughtful about whether each new application or idea really needs a whole new model. If we really do have major architectural improvements that preclude directly reusing model weights, then we should invest in more aggressive forms of distillation that allow us to retain as much of the previous generation’s capabilities as possible.

Large models aren’t more capable in the long run if we can iterate faster on small models

LoRA updates are very cheap to produce (~$100) for the most popular model sizes. This means that almost anyone with an idea can generate one and distribute it. Training times under a day are the norm. At that pace, it doesn’t take long before the cumulative effect of all of these fine-tunings overcomes starting off at a size disadvantage. Indeed, in terms of engineer-hours, the pace of improvement from these models vastly outstrips what we can do with our largest variants, and the best are already largely indistinguishable from ChatGPTFocusing on maintaining some of the largest models on the planet actually puts us at a disadvantage.

Data quality scales better than data size

Many of these projects are saving time by training on small, highly curated datasets. This suggests there is some flexibility in data scaling laws. The existence of such datasets follows from the line of thinking in Data Doesn’t Do What You Think, and they are rapidly becoming the standard way to do training outside Google. These datasets are built using synthetic methods (e.g. filtering the best responses from an existing model) and scavenging from other projects, neither of which is dominant at Google. Fortunately, these high quality datasets are open source, so they are free to use.

Directly Competing With Open Source Is a Losing Proposition

This recent progress has direct, immediate implications for our business strategy. Who would pay for a Google product with usage restrictions if there is a free, high quality alternative without them?

And we should not expect to be able to catch up. The modern internet runs on open source for a reason. Open source has some significant advantages that we cannot replicate.

We need them more than they need us

Keeping our technology secret was always a tenuous proposition. Google researchers are leaving for other companies on a regular cadence, so we can assume they know everything we know, and will continue to for as long as that pipeline is open.

But holding on to a competitive advantage in technology becomes even harder now that cutting edge research in LLMs is affordable. Research institutions all over the world are building on each other’s work, exploring the solution space in a breadth-first way that far outstrips our own capacity. We can try to hold tightly to our secrets while outside innovation dilutes their value, or we can try to learn from each other.

Individuals are not constrained by licenses to the same degree as corporations

Much of this innovation is happening on top of the leaked model weights from Meta. While this will inevitably change as truly open models get better, the point is that they don’t have to wait. The legal cover afforded by “personal use” and the impracticality of prosecuting individuals means that individuals are getting access to these technologies while they are hot.

Being your own customer means you understand the use case

Browsing through the models that people are creating in the image generation space, there is a vast outpouring of creativity, from anime generators to HDR landscapes. These models are used and created by people who are deeply immersed in their particular subgenre, lending a depth of knowledge and empathy we cannot hope to match.

Owning the Ecosystem: Letting Open Source Work for Us

Paradoxically, the one clear winner in all of this is Meta. Because the leaked model was theirs, they have effectively garnered an entire planet’s worth of free labor. Since most open source innovation is happening on top of their architecture, there is nothing stopping them from directly incorporating it into their products.

The value of owning the ecosystem cannot be overstated. Google itself has successfully used this paradigm in its open source offerings, like Chrome and Android. By owning the platform where innovation happens, Google cements itself as a thought leader and direction-setter, earning the ability to shape the narrative on ideas that are larger than itself.

The more tightly we control our models, the more attractive we make open alternatives. Google and OpenAI have both gravitated defensively toward release patterns that allow them to retain tight control over how their models are used. But this control is a fiction. Anyone seeking to use LLMs for unsanctioned purposes can simply take their pick of the freely available models.

Google should establish itself a leader in the open source community, taking the lead by cooperating with, rather than ignoring, the broader conversation. This probably means taking some uncomfortable steps, like publishing the model weights for small ULM variants. This necessarily means relinquishing some control over our models. But this compromise is inevitable. We cannot hope to both drive innovation and control it.

Epilogue: What about OpenAI?

All this talk of open source can feel unfair given OpenAI’s current closed policy. Why do we have to share, if they won’t? But the fact of the matter is, we are already sharing everything with them in the form of the steady flow of poached senior researchers. Until we stem that tide, secrecy is a moot point.

And in the end, OpenAI doesn’t matter. They are making the same mistakes we are in their posture relative to open source, and their ability to maintain an edge is necessarily in question. Open source alternatives can and will eventually eclipse them unless they change their stance. In this respect, at least, we can make the first move.

Marisa Tomei

CategoriesIssue 2023Q2, Movies to Watch

Marisa Tomei  , italian, born December 4, 1964)[1] is an American actress. She was a cast member on the Cosby Show spin-off A Different World in 1987. For her role in the 1992 comedy My Cousin Vinny, she won the Academy Award for Best Supporting Actress. She has received two additional Oscar nominations for In the Bedroom (2001) and The Wrestler (2008), with the latter also earning her nominations at the BAFTA and Golden Globe Awards.

Tomei has appeared in a number of notable films, including Chaplin (1992), The Paper (1994), What Women Want (2000), Before the Devil Knows You’re Dead (2007), The Ides of March (2011), Crazy, Stupid, Love (2011), Parental Guidance (2012), Love Is Strange (2014), and The Big Short (2015). She also portrayed May Parker in the Marvel Cinematic Universe (MCU), having appeared in Captain America: Civil War (2016), Spider-Man: Homecoming (2017), Avengers: Endgame (2019), Spider-Man: Far From Home (2019), and Spider-Man: No Way Home (2021).

Tomei was formerly involved with the Naked Angels Theater Company and appeared in Daughters (1986) before making her Broadway debut in Wait Until Dark (1998). She earned a nomination for the Drama Desk Award for Outstanding Featured Actress in a Play for her role in Top Girls (2008), and a special Drama Desk Award for The Realistic Joneses (2014). She returned to Broadway in the revival of The Rose Tattoo in 2019.

Strange Things Volume III: The Dying Banks and the Singularity

CategoriesGamestop_, Issue 2023Q2

A new financial crisis is brewing. Last month, 4 major banks collapsed or were shut down, and this past weekend First Republic Bank was seized by the FDIC and sold in a fire sale to JP Morgan Chase. There is an accelerating withdrawal of money throughout the entire system.

The cracks are widening, and Strange Things are going on in the world of banking. The gravitational fields made by the Fed to avoid prior crises are now creating a new crisis. Anything will be done to paper up the disemboweled banks bleeding from the latest hiking cycle.

Welcome to the Singularity.


r/Superstonk - the Singularity
the Singularity

Silicon Valley Bank (SVB) was a commercial bank that provided financial services to technology and life science companies, as well as venture capital and private equity firms. Founded in 1983 in Santa Clara, California, the bank had expanded to serve clients in major innovation hubs across the world, including New York, Boston, London, and China. Silicon Valley Bank was known for its expertise in the technology and life science industries, providing tailored solutions to help companies and investors navigate complex financial landscapes.

To incentivize companies to stay with them, SVB would offer a range of financial products, and include bonus “gifts” such as free subscriptions to many of the essential SaaS services that startups need (Salesforce, for example.) More insidiously, however, the bank offered to help firms raise additional capital if they stayed with the bank, and kept this money in their account.

This is eerily reminiscent of Mafia rackets, where businesses were given incentives to keep a gang as their business partner in a money laundering scheme.

As a result of these policies, Silicon Valley Bank had a unique customer base- almost entirely high end VC, PE and startup clients who held millions of dollars in each deposit account.

Silicon Valley Bank, like any bank, is constrained by a variety of regulations that limit the types of investments it can make- loans and bonds, especially “Tier 1” HQLA (High-Quality Liquid Assets), would make up the majority of its balance sheet.

During 2021 and the first quarter of 2022, the Fed had been plowing $120B a month into the market via QE, and interest rates were suppressed near the zero bound. This created a massive influx of capital- deposits at SVB ballooned from $61bn at the end of 2019, to a peak of $174bn at the end of 2022.

With limited places to put these funds, SVB had poured them all into Treasuries and MBS in hopes of remaining compliant with federal regulations.

We can see their balance sheet below:


r/Superstonk - SVB Balance Sheet (Consolidated)
SVB Balance Sheet (Consolidated)

However, this would soon come back to haunt them.

While digging through their financials, I found something startling. Their assets were segregated into two different types: AFS and HTM. AFS stood for Available for Sale, these were assets that were liquid, marked to market (meaning that if there were losses, they would be counted as unrealized losses on the BS). HTM stood for Hold to Maturity- these were bonds and MBS that would be held until the maturity date of the instrument.

Strikingly, HTM securities were not hedged for interest rate risk and did NOT have to be marked to market. They assumed that the risk profile for these bonds was ZERO.

r/Superstonk - Credit Risk of HTM is 0
Credit Risk of HTM is 0


What was even more terrifying is I soon found out that this is an industry standard practice- SVB is not alone. Any bank chartered in the US, if it holds HTM securities, does not have to record an ECL (Expected Credit Losses) on them and thus will not hold any cash in reserve, or hedge against the security falling in value!

Here’s a further breakdown. They held billions in MBS, CMBS, and even variable-rate CMO- Collateralized Mortgage Obligations.

r/Superstonk - SVB Assets breakdown
SVB Assets breakdown


All that for a drop of blood. The average yield on all securities was a measly 1.56%.

r/Superstonk - Average Yield
Average Yield

They had plowed billions of dollars worth of deposits into these securities at ultra-low interest rates, and as the Fed began its hiking cycle, a vicious problem began to confront them.

Debt securities trade inverse to the interest rates on them- so the higher the Fed hiked, the more the market value fell. For a while, this was managed fine as they kept receiving deposit inflows.

However, late in 2022, some VCs began to get worried and warned their companies to begin pulling out of SVB.

The Fed’s hiking cycle caused billions of dollars in unrealized losses on their balance sheet, with around $22B coming from AFS securities- however, this was only part of the picture as HTM securities did not have to be marked down.

Like any bank, they are fractionally reserved- $14B of cash deposits and cash equivalents backed up $173B of deposit liabilities.

However, this figure is misleading as it includes other securities. When I looked closer, they only had $2.3B of actual cash on hand.

This process accelerated in January and February. They ran out to raise capital, but the markets smelled a corpse. The capital raise failed and on March 9th the stock collapsed 62%.


r/Superstonk - SVB March 9th, 2023
SVB March 9th, 2023

During the next 24 hours, 85% of SVB’s bank deposits were withdrawn or attempted to be withdrawn.

That’s the fastest bank run in history.

By the end of Friday, March 10th, they would be in FDIC receivership and the bank would be closed.


Within the month of March, Silvergate, Silicon Valley, Signature, and Credit Suisse would all collapse. First Republic would fall in late April, and PacWest now stands at the brink.

The problem that plagued these banks was a different beast than 2008- instead of making bad loans, they had made bad investments. The Fed had promised infinite liquidity without repercussions, and the risk management committees, bound by regulation, had followed the rest of the banking sector headlong into bonds when the prices were at their highest.

Now, with inflation still raging and the Fed stating they are “unfinished” with the hiking cycle, the banking sector has a massive gaping hole blown through it.


r/Superstonk - Unrealized losses at banks
Unrealized losses at banks

According to the chair of the Federal Deposit Insurance Corporation (FDIC), there were $620 billion of such unrealized (or paper) losses sitting on U.S. bank balance sheets in early March.

However, this does not account for all securities. More sober estimates put this figure closer to $1.7T dollars! (See here)


r/Superstonk - $1.7T of Unrealized Losses
$1.7T of Unrealized Losses

Banks as a whole have been using the HTM loophole to shift more and more securities into this designation, in order to avoid mark-to-market losses on their books. At the same time, they’ve reduced the amount of AFS securities.

HTM securities also are not allowed to be hedged.

Which means that none of these bonds have been hedged for interest rate risk. Even if they were allowed to do so- what would that change? The system as a whole would want to hedge the trillions of dollars of interest rate risk they carry, and who would take the other side of that trade?

If any firm did, they would face the same fate as AIG did during the 2008 Financial Crisis…


r/Superstonk - Increasing amounts of HTM held at banks
Increasing amounts of HTM held at banks

Silicon Valley then, is not unique. In a startling research paper entitled Monetary Tightening and U.S. Bank Fragility in 2023, the authors made several terrifying points:


r/Superstonk - The entire system is at risk
The entire system is at risk

They then continued:

“Marking the value of real estate loans, government bonds, and other securities results in significant declines in bank assets. … The median value of banks’ unrealized losses is around 9% after marking to market. The 5% of banks with the worst unrealized losses experience asset declines of about 20%. We note that these losses amount to a stunning 96% of the pre-tightening aggregate bank capitalization.”

There are 190 banks across the US, with $300B of deposits, that are at substantial risk of failure.

The entire banking system is at risk. At first, the deposit flight was simply out of the small commercials and into the bulge brackets, the large prime banks that are the “Too Big to Fail” institutions from the 2008 financial crisis.

But now, the deposit flight is widespread. Hundreds of billions of dollars of deposits are missing from the banking system, even the prime banks- where did they go? (See here)


r/Superstonk - hundreds of billions are missing from the banking system
hundreds of billions are missing from the banking system

One of the primary beneficiaries has been the shadow banks- the opaque financial institutions that can take on deposits and lend them out through the monetary plumbing that underlies the system.

Money Market Funds, for example, have seen $640B in inflows since the end of last year. In an ill-fated attempt to prevent collateral shortages in the shadow banking system, the Fed opened up the Reverse Repo window to allow MMFs and banks to park their cash overnight and hold Treasuries as collateral.


r/Superstonk - Cash flowing into MMFs
Cash flowing into MMFs

This was discussed in-depth in my DD on MMFs here: (Major Signals Flashing Code Red in the Shadow Banking System, RRP Hitting $1T is just the tip of the Iceberg) (August 4th, 2021).

The cash parked in the RRP window has held above $2T now for months, and the award rate (the interest rate paid on RRP cash deposited) has been steadily increasing, and stands at a record 4.8% as of writing.


r/Superstonk - Interest rate paid on RRP
Interest rate paid on RRP

The MMFs are therefore able to offer attractive rates, often in excess of 4%, while the banks are confined to near 0% interest on deposits.

This financial gravity created by the Fed’s RRP is sucking cash out of the banking system and into the shadow banks, at the same time that the traditional banks are bleeding from the hole blown through them via their bond portfolios.

Just the other week, Apple announced a new high-yield savings account, paying a shocking 4.15%, and this product is to be managed by Goldman Sachs.

This move has contributed to over $60B of outflows from big US financial groups such as Charles Schwab, State Street and M&T.


r/Superstonk - Deposits getting sucked out of US Banks
Deposits getting sucked out of US Banks

(See here)

Why hold deposits when you can plow funds into a shadow bank and hold positive yielding Treasuries instead?

The system is being drained. With Treasuries finally providing higher rates, at a “risk-free” yield, the Fed and Treasury combined have essentially created a massive money laundering scheme via the banks.

r/Superstonk - Peruvian Bull Tweet, March 14th 2023
Peruvian Bull Tweet, March 14th 2023

In a fractional reserve banking system, they only have a few percent of the deposits as the actual cash on hand- so it doesn’t take that much to push many of these firms over the edge.

The FDIC, the supposed savior of the system, is a dead man walking- the Deposit Insurance Fund (DIF) balance was $128.2 billion on December 31, 2022, up $2.8 billion from the end of the third quarter. The reserve ratio increased by one basis point to 1.27 percent as insured deposits increased 1.4 percent.

This fund exists to back up $19 TRILLION of deposit liabilities throughout the American financial system.

The worst part? The dominos will continue to fall as the gravitational pull rips more banks into pieces. Now, the failure of the latest firm, First Republic, has put the total failure amount (adjusted to inflation) HIGHER THAN 2008.

And that’s not even counting Silvergate or Credit Suisse!


r/Superstonk - Bank Failures by year, 2023 already largest on record
Bank Failures by year, 2023 already largest on record

As the fallout continues from the most disastrous Fed policy error in a century, only one question remains to be asked: Who will be left to hoover up the wreckage? Only the big boys like JP Morgan, who was announced this morning as the winning bidder for First Republic.

Desperate to prevent a widespread bank collapse like the 1930s, the Fed will heap increasing quantities of liquidity onto the system. The prime banks will swallow more and more assets, growing ever larger.

As the system moves beyond the event horizon, the money backing ALL liabilities will move to Infinity.

The Fed has created a singularity from which there is no escape.



r/Superstonk - Singularity


Thanks for reading!

You can follow me on Twitter here: https://twitter.com/peruvian_bull