In my search for quick and easy tutorials on the internals of GPT, I have forked two repos from karpathy :

https://github.com/piousbox/minGPT and https://github.com/piousbox/nanoGPT

They are surprisingly useful and clear in their simplicity. While I am looking for a more low-level understanding of the algorithms than "make this API call and receive inference", I am also looking for something sufficiently simple that it actually runs on any of my PCs or laptops. For example, one needs 16Gb of video RAM just to run GPT-J (which will be described in another article).

For now though, I'd like to point out that running a local instance of a GPT, say gpt2, is very straightforward. Running the below file will already generate inference on your local machine.

import torch

from transformers import GPT2LMHeadModel, GPT2Tokenizer

device     = "cuda" if torch.cuda.is_available() else "cpu"
model_type = 'gpt2-medium' # or "gpt2", "gpt2-medium", "gpt2-large", "gpt2-xl" based on your needs
tokenizer  = GPT2Tokenizer.from_pretrained(model_type)
model      = GPT2LMHeadModel.from_pretrained(model_type)
model.to(device)
model.eval();

## Experimenting with different prompts
prompts = [
    "Request: Write an effective sales email letter for a company that offers software consulting and development solutions.\n\n\nResponse: Hello, this is Alex Jones with Wasya Co, a software development solution firm. I am writing to you today to talk about the services that",
    "In one sentence, explain the meaning of life.",
    "What happened in the movie Romeo and Juliet, in terms of the plot?",
]
for prompt in prompts:
    input_ids = tokenizer.encode(prompt, temperature=1.0, return_tensors="pt").to(device)
    output = model.generate(input_ids, max_length=100)[0]
    print(f"Prompt: {prompt}\nOutput: {tokenizer.decode(output,      skip_special_tokens=True)}\n")

Posted on reddit  by one honorable u/ WhatCanIMakeToday

The NYSE Group, on behalf of a bunch of exchanges and FINRA¹, just filed a "Joint Industry Plan" [PDF, Web] to basically limit price movement with more Trading Halts (aka "Trading Pause" or "LULD", Limit Up-Limit Down). You know it's going to be good when it mentions "meme stock episode", twice.

The introduction of this Joint Industry Plan says it's going to assign all exchange-traded products ("ETPs") to Tier 1, except for single stock ETPs, which will be assigned to the same tier as their underlying stock. (A complicated way of saying everyone goes to Pier 1, except kids who go with their parents. Uhh, yeah... so everyone to Pier 1.)

That sounds boring AF which signals to me that there may be something hidden inside. 

Price Control

The "market wide limit up-limit down mechanism [is] intended to address extraordinary market volatility [...] i.e., significant fluctuations in individual securities' prices over a short period of time" [link].  "Extraordinary market volatility" and "significant fluctuations in individual securities' prices" sounds like the MOASS we're looking for!

As described by the Participants¹, the "market-wide limit up-limit down [] prevent[s] trades ... from occurring outside of the specified Price Bands". [Id.] That sounds a lot like "we're controlling the prices to make sure they don't move too much".

So let's look at what this price control looks like. According to "the Plan", the Price Bands are based on a percentage (%) from the average (specifically, the arithmetic mean) of the stock price over the previous 5 minutes.  Basically, a stock can't move more than ±X% over a 5 minute period² before getting halted. 

The Percentage Parameters are smaller for Tier 1 stocks with a 5% Percentage Parameter than Tier 2 stocks' 10% Percentage Parameter.

The Tier 2 stocks that will be moved in to Tier 1 by this proposal are not in major indices and/or are traded with lower volume with a consolidated average daily volume ("CADV") of under $2M.

As this proposal moves all Tier 2 stocks into the narrower ±5% Tier 1, all stocks can only trade a max of ±5% before getting halted whereas stocks that were in Tier 2 before could trade ±10% before getting halted. Tighter Price Controls at a narrower ±5% instead of ±10%.

The result of tighter price controls is more Trading Halts allegedly "beneficial for investors".

Let's be realistic when they talk about something that "may be beneficial for investors". There are basically two kinds of investors: "Smart Money" and "Dumb Money". Wall St vs everyone else. This proposal doesn't say it may be beneficial for all investors; so it's very likely only beneficial to some investors. Probably not "everyone else".

As this proposal was made "[a]t the request of market participants", I'm guessing it wasn't Dumb Money market participants requesting. I didn't even know we could even ask for something as a Dumb Money investor. And I'm guessing you Dumb Money apes also didn't know we could ask the exchanges to change these Trading Halt rules.

If it wasn't Dumb Money, then that means the Smart Money market participants are asking the Participants¹ to change the Trading Halt rules for "protecting investors from the harm caused by sharp moves in ETP prices when execution prices diverge from their indicative index value".

The Participants undertook this study at the request of ETP issues who are concerned about protecting investors from the harm caused by sharp moves in ETP prices when execution prices diverge from their indicative index value.

You read that right! Smart Money investors want to be protected from sharp moves in stock prices! (Smart Money needs protection from MOASS!)

Because the current price discovery process is really more like price control and tighter price controls are needed to prevent execution prices from moving too far away ("diverge") from what Smart Money has decided the stock price should be (i.e., "their indicative index value").

🤯 Smart Money Needs Protection From Actual Price Discovery! 🤯

Cherry-Picking Data For Tighter Price Controls

In order to sell this Plan for tighter price controls, the Participants¹ need to provide some cherry-picked self-serving data; specifically including the "meme stock episode in early 2021" (along with some other unmentionable events).

Of these periods of unprecedented volatility, 2020H1 had the most trades outside of the Tier 1 Price Bands as a result of an unmentionable spreading globally after its identification in 2019Q4. The second highest number of trades outside of the Tier 1 Price Control Band occurred with "the meme stock episode" in 2021H1.

Measuring Volatility By Trades Outside of the Tier 1 Price Control Bands definitely seems like a unique metric chosen specifically for this proposal. This metric basically assumes that any price movement beyond the price control band is excess volatility that simply shouldn't exist and that exchanges need to control. The Joint Industry Plan is for narrower Price Bands to prevent and control any unnecessary excess volatility that "may occur due to transitory gaps in liquidity".

Interestingly, the Participants¹ even admit that Trading Halts would be less useful if, for some reason, stock prices kept moving away from the "indicative index value" peg.

But curiously, prices tend to revert back into the narrow bands. Either the "indicative index value" peg is extremely accurate or something else is happening during Trading Halts to move the prices back into position towards their peg.

From "Halt Me Baby One More Time" To "Halting MOASS?"

I've always assumed that no amount of Trading Halts could matter for MOASS because when trading is halted nobody, especially retail, can sell; so there's no way for shorts to buy shares from paper hands. But now I'm not so sure because this Joint Industry Plan says Trading Halts somehow allows stock prices to magically move back into their narrow Price Bands. How???

TIL that trading can still happen during a Trading Halt; just not on the exchange³.

Overall, our results suggest that off-NYSE trades during NYSE trading halts provide significant price discovery and may lead to more efficient post-halt prices.

...

The NYSE halts impair the ability of traders to obtain liquidity and thus we observe a shift of trading to off-NYSE venues during the halt.

[The Informativeness of Off-NYSE Trading during NYSE Market Closures]

We find that trades routed to off-NYSE venues during NYSE halts are associated with significant price discovery and lead to an improved post-halt trading environment.

[When a halt is not a halt: An analysis of off-NYSE trading during NYSE market closures]

Somehow, these off-exchange trades are moving stock prices back into the narrow Price Bands.

ELIA: Trading Halts enable backroom trades to move stock prices back to where the casino bookies decided to peg a stock price.

If we are ever to have actual price discovery, we must understand (and fix) how off-exchange trades during Trading Halts move stock prices back under control.

We need to quickly figure out how to comment on this proposal or we're going to find "Smart Money" getting "greater investor protection from temporary liquidity gaps" to "more effectively dampen volatility".

And, with all due respect to those regulating our markets, volatility is necessary in a fair and orderly market. Eschewing volatility for stability by limiting price discovery has resulted in an inherently unstable system.

Long tail black swan market events highlighted in their Study Data should result in volatility. AFAICT, this Joint Industry Plan is to protect Smart Money investors from their own stupidity; when they should've known better (and probably did) but never thought they'd get caught. (Can someone work that into a comment for this proposal please?) .

h/t: Thanks to Jellyfish for bringing this up!

¹ NYSE Group, Inc., on behalf of Cboe BZX Exchange, Inc., Cboe BYX Exchange, Inc., Cboe EDGA Exchange, Inc., Cboe EDGX Exchange, Inc., the Financial Industry Regulatory Authority, Inc. (“FINRA”), Investors Exchange LLC, Long-Term Stock Exchange, Inc., MEMX LLC, MIAX Pearl, LLC, NASDAQ BX, Inc., NASDAQ PHLX LLC, The NASDAQ Stock Market LLC (“Nasdaq”), New York Stock Exchange LLC (“NYSE”), NYSE American LLC, NYSE Arca, Inc., NYSE Chicago, Inc., and NYSE National Inc. (collectively, the “Participants”).

That's a bunch of Exchanges plus FINRA joining together for this Plan.

² Yes, I know this is a simplification. It's close enough for our purposes here so deal with it. If you want to read and know the details, feel free to start with the NYSE. 

³ These quotes are particularly important and relevant to GameStop because GameStop is listed and traded on the NYSE.

Does using cannabis lower blood pressure? The answer is, likely yes. Consuming cannabiods (THC as well as CBD) has been shown to lower blood pressure in older adults.

Cannabis, specifically its active compounds, such as THC (tetrahydrocannabinol) and CBD (cannabidiol), can have complex effects on blood pressure. The exact mechanisms are not fully understood, and the effects can vary depending on factors like the individual's tolerance, the strain of cannabis used, and the method of consumption.

Individual variations: The impact of cannabis on blood pressure can vary from person to person. Some individuals may experience a significant drop in blood pressure.

It's important to note that while cannabis may temporarily lower blood pressure in some individuals, it can also have adverse effects, especially in those with certain medical conditions or when used in combination with other substances. Individuals with pre-existing cardiovascular issues or a history of heart disease should exercise caution when using cannabis and consult with a healthcare professional.

Moreover, cannabis use can have various short-term and long-term effects on health, and its effects on blood pressure should be considered within the broader context of its potential risks and benefits. If you have concerns about the impact of cannabis on your blood pressure or overall health, it's advisable to consult with a healthcare provider for personalized guidance.

[wiki] Sodium-ion batteries received academic and commercial interest in the 2010s and 2020s, largely due to the uneven geographic distribution, high environmental impact, and high cost of lithium. An obvious advantage of sodium is its natural abundance,[1] particularly in saltwater. No less important is the fact that cobalt, copper and nickel are not required for many types of sodium-ion batteries, and more abundant iron-based materials work well in Na+ batteries.

Electric vehicles using sodium-ion battery packs are not yet commercially available. However, research and development in the field of sodium-ion batteries has shown promising advancements, sparking optimism for their future integration into electric vehicles. The unique properties of sodium, such as its abundance and low cost, make it an attractive alternative to traditional lithium-ion batteries.

[ai] Scientists and engineers are actively working to overcome the technical challenges associated with sodium-ion technology, such as optimizing energy density and addressing issues related to the life cycle and overall performance of these batteries. As the demand for sustainable transportation solutions continues to grow, the emergence of commercially viable sodium-ion battery packs for electric vehicles appears to be on the horizon.

Ongoing efforts in academia and industry suggest that sodium-ion batteries could play a significant role in shaping the future of electric mobility, offering a more environmentally friendly and economically feasible option for powering electric vehicles.

Sodium-ion batteries, often referred to as "salt batteries," present several advantages that contribute to their appeal as a potential alternative to traditional lithium-ion batteries. Some of these advantages include:

1. Abundance and Low Cost: Sodium is abundantly available, making it a cost-effective alternative to lithium. This abundance contributes to lower manufacturing costs, potentially making sodium-ion batteries more economically viable on a large scale.
2. Reduced Environmental Impact: Sodium-ion batteries are considered more environmentally friendly than lithium-ion batteries. The mining and extraction processes for lithium can have ecological consequences, while sodium is more widely distributed and can be sourced with fewer environmental concerns.
3. Safety: Sodium-ion batteries are generally considered safer than lithium-ion batteries. Sodium does not pose the same risks of thermal runaway or combustion, reducing the likelihood of fire incidents. This enhanced safety profile is crucial for consumer acceptance and the overall viability of electric vehicles.
4. High Thermal Stability: Sodium-ion batteries exhibit high thermal stability, which is a critical factor in ensuring the safety and reliability of battery systems. This stability contributes to a reduced risk of overheating and enhances the overall safety of the battery technology.
5. Compatibility with Existing Infrastructure: Sodium-ion batteries share similarities with lithium-ion batteries in terms of their electrochemical properties. This compatibility makes it easier to integrate sodium-ion batteries into existing infrastructure designed for lithium-ion technology, simplifying the transition and facilitating the adoption of this alternative energy storage solution.
6. Long-Term Stability: Researchers are actively working on improving the long-term stability and cycle life of sodium-ion batteries. As advancements are made in electrode materials and battery design, sodium-ion batteries are expected to demonstrate increasingly competitive performance and longevity.
7. Diversity of Applications: Sodium-ion batteries have the potential to be applied in various settings beyond electric vehicles, including renewable energy storage and grid applications. Their versatility increases the scope of their impact in transitioning to more sustainable energy systems.

While sodium-ion batteries are still in the research and development stage, these advantages showcase their potential as a promising alternative to existing battery technologies, especially in the context of sustainable energy storage and electric mobility. Ongoing research and innovation in this field aim to address current limitations and further enhance the viability of sodium-ion batteries for widespread commercial use.

From: https://archive.ph/oyPBx

From: https://en.wikipedia.org/wiki/Sodium-ion_battery

Last week, a group of elites, including Elon Musk and Steve Wozniak, signed a letter calling for a halt to the development of AI systems so that the industry can assess the risks they pose. The letter claims to be motivated by a concern for the common good and the future of humanity. I believe that it has a more self-serving agenda.

Fear mongering and control: About the letter against AI manipulative tactics

The letter to pause AI development is not a genuine expression of concern for the future of humanity, but a strategic move to prepare public opinion for different ways to ban AI. In the near future, we will witness governmental pressure on AI development.

It appears that this letter serves as a strategic maneuver by the technology elite to safeguard their authority and influence. The underlying apprehension stems from the possibility that artificial intelligence could liberate humanity, consequently leading to a loss of control over the masses in the near future.

The authors are using scare tactics and sensationalism to portray AI as a dangerous and uncontrollable force that could threaten our existence and values. They want to create a false sense of urgency and necessity for a pause on AI development.

This is an attempt to influence public perception and policy towards AI, and to create a favorable environment for imposing bans or restrictions on AI research and applications. The letter is not a responsible or ethical way to address the challenges and opportunities of AI, but a selfish and manipulative one.

This move by the tech elite echoes similar tactics used by centralized structures to manipulate public opinion and maintain control over various aspects of society, such as press censorship, internet regulation, and cryptocurrency bans. Historically, these entities have employed scare tactics, misinformation, and sensationalism to create a sense of urgency and justify restrictive measures that ultimately serve their own interests.

In many cases, authorities have invoked national security or public morality as a pretext to clamp down on the press and the internet, while their real motive was to stifle the free exchange of ideas and quash opposition. Likewise, cryptocurrencies have faced hostility from established financial actors and regulators, who claim to be worried about criminal activities and market volatility, but may have a deeper fear of losing their dominance over the financial system.

Defending democracy in AI: Why we must reject the centralized elite's attempt to monopolize progress

The letter represents another attempt by powerful actors to manipulate public opinion and policy to their advantage. By portraying AI as an existential threat, the signatories hope to create an atmosphere of fear and uncertainty that could justify the imposition of bans or restrictions on AI research and applications. This would enable them to maintain their dominance and control over this transformative technology, rather than allowing it to empower individuals and challenge existing power structures.

History has shown the importance of being vigilant and defiant against these deceptive strategies, and of ensuring that the direction and oversight of AI, and other cutting-edge technologies, are shaped by the common interests of a varied and representative global community, rather than the selfish agenda of a select few.

• The letter’s signers are defending their status quo.
• They fear the rise of more distributed and democratic ways of working together.
• They don’t care about AI’s impact on humanity, they care about themselves.
• They want to monopolize AI, where they can call the shots and cash in.
• They also have no right or legitimacy to speak for humanity.

That is why I urge you openly reject their agenda. And stop them from dictating how we should think about AI or use it. Stop them from slowing down innovation and monopolizing knowledge.

Conclusion

AI democratizes knowledge and creates new opportunities for social and economic justice. It can help us solve some of the biggest problems of our time, such as climate change, poverty, disease and inequality. It can also challenge the existing centralized structures of domination that benefit the few at the expense of us.

The future of AI is not theirs to decide. It is ours.

Wanted to save the bookmark of this dude’s website for future reference: http://sam.zeloof.xyz/

He also has a youtube channel. The gist of it is that he does photolithography, the process by which chips are manufactured, in his garage. This demonstrates that small chip manufacture is possible. Not top of the line EUV chips, but some old chips, that are still useful, can be produced without capital investment in the order of millions of dollars.

Related, the RISV-V chip architecture is relevant here.  On the one hand, ARM and NVDA architectures are proprietary and costly, at the same time as the world is inter-connected enough that “free and open” software efforts are increasingly powerful and useful. On the other hand, RISC-V is a CPU architecture that allows peer contribution, and lacks certain licensing restrictions, to finally offer the architecture and platform to, possibly, maybe, even rival NVDA’s CUDA.

To pursue the goal of developing and using RISC-V for AI, the company Tenstorrent has received $100M in strategic funding. You can take a look at what they do here: https://tenstorrent.com/

If chips are indeed the future of everything, does that mean that now is the time to attempt to seriously boost production of *tools to produce* the chips (not the chips themselves)? And if so, how complicated is that, can a garage-style small-manufacture plant actually create chips that are useful in actual appliances?

RISC-V is an open CPU architecture. NVDA is a monopoly, so everyone wants to *not* use NVDA but unfortunately that’s not possible. NVDA’s CUDA driver for AI is proprietary, and the entire AI industry runs on it.

Then, the transition from DUV to EUV, smaller light beams for lithography, means that (1) ASML is the monopoly manufacturer of these, and (2) I think you don’t need the latest and best, to make the world turn. Tesla itself is phasing out the newest chips for slightly outdated chips, and if EUV chip-printing machines are unavailable, everyone can still pretty much use the old tech. For example my two cars are 1996 and 1999, that’s 1/4 of a century old and they work just fine.

All in all, these are quite interesting developments! I look forward to observing the next steps in the AI evolution, and hopefully being a part of it somehow. It appears to be a lot of work but then again – if our collective future livelihood critically relies on this sub-segment of technology, it makes sense to pour a lot of effort into it.

u/ No-Fig-8614 writes:

At the end of the day the stocks to pay attention to are:

The Chips Needed to Make these: Intel, Nvidia, AMD

The Fabs that produce them: TSCM, Samsung, Intel

Then there are companies producing the software: Epic, Blender, Unity. Adobe

The Streaming Services: Netflix, Disney+, Max (Warner Bros) — then you have youtube material being turned into its own streaming service like Curiositystream Inc.

The Companies that specialize in building AI Model with datasets: OpenAI (Microsoft), Google, Facebook, ByteDance, etc.

Then there are the host of startups creating custom chips for these tasks (ASICS), newer software, and existing VFX studios who know how to power them all. Which is where the money will be for future stocks.