Marc Garcia (@datapythonista) header
Marc Garcia (@datapythonista) profile picture

pandas core developer

Joined April 2022
Photo by Marc Garcia (@datapythonista)Photo by Marc Garcia (@datapythonista)Photo by Marc Garcia (@datapythonista)Photo by Marc Garcia (@datapythonista)Photo by Marc Garcia (@datapythonista)Photo by Marc Garcia (@datapythonista)Photo by Marc Garcia (@datapythonista)Photo by Marc Garcia (@datapythonista)Photo by Marc Garcia (@datapythonista)
Marc Garcia retweeted
The early design decisions for the Categorical type were under strain because of our streaming engine. Every data chunk carried its own mapping between the categories and their underlying physical values, forcing constant re-encoding. The global StringCache we built to solve it caused lock contention and wasn't designed for a distributed architecture. The new Categories object, released in 1.31, solves this, and gives you: • Control over the physical type (UInt8/16/32) • Named categories with namespaces • Parallel updates without locks • Automatic garbage collection When you know the categories up front you can use Enums. They're faster because of their immutability and allow you to define the sorting order of values. The StringCache is now a no-op, but the code will keep working how it used to (with global Categories). You can also migrate by replacing it with explicit Categories where needed. The result is a Categoricals data type that works well on the streaming engine without performance degradation, and is compatible with a distributed architecture. Read the full deep dive: pola.rs/posts/categoricals-r…
1
9
64
5,582
Marc Garcia retweeted
DataFusion 52 is released datafusion.apache.org/blog/2…
11
68
3,948
Marc Garcia retweeted
We're happy to announce the release of #pandas 2.3.0. You can install it with `pip install pandas` or `conda install -c conda-forge pandas`. Thanks to all contributors and sponsors who made this release possible! The release notes can be found at: pandas.pydata.org/docs/whats…
3
8
1
74
6,691
Tech occupations have been among the hardest hit by steep declines in hiring since 2018, per BI;
75
85
12
407
122,559
Possibly the greatest single male athletic performance of all time
3,170
34,394
3,425
418,982
28,537,009
Marc Garcia retweeted
Today we are launching the first open Crash Course training sessions with a limited time discount. These instructor-led sessions are open to everyone looking to get up and running with Polars. Find a date and sign up via our Academy: pola.rs/academy/
1
8
1
60
4,583
Ella lo único que quiere es cantar:
192
2,499
217
18,864
581,544
Duck said give me my pookie back 🥹🥰
21
492
13
5,924
116,437
I hope these two never separate 😭❤️
244
7,756
297
66,931
3,457,860
This made my day! 😀
342
1,289
195
8,790
537,974
The reason why i love cats
1,204
7,255
1,158
67,197
6,061,925
Marc Garcia retweeted
Perfect training partner
89
815
65
9,387
252,834
why are cats so silly
454
15,675
712
160,141
11,829,143
Marc Garcia retweeted
I've written a blog about Tonbo's research on async Rust and io_uring: tonbo.io/blog/async-rust-is-… We need to be careful to avoid the cancellation problem when using async Rust and io_uring together.
4
42
4
281
30,392
Trying to impress cats and failing is universal
50
466
10
5,967
162,890
.@jimmy_dore: "Barack Obama made sure the bankers got their bonuses while he kicked 5 million families out of their homes. He got more money from Wall Street than John McCain did. There was a Citigroup email with a list of people they wanted in his cabinet. Every person on that list ended up in his cabinet. That got released by Wikileaks, and that's why they've been trying to kill Julian Assange ever since. Obama wasn't a departure from George W. Bush. Just his skin color. He took us from two wars to seven. He gave us Mitt Romney's right-wing health care plan that came out of the Heritage Foundation, which is the same people that gave you Project 2025. He dropped more bombs than George Bush, and nobody noticed, and that's why the donor class wanted to have Kamala Harris. They thought we'd have a female Barack Obama, dropping bombs, stealing resources, and doing neoliberal and neoconservative policies, but then she couldn't get a vote in the primary because she has no political talent. So then they forced her on Joe Biden as the Vice President, and then, of course, they did a coup just like Jim Gaffigan said at the Al Smith dinner. After Occupy Wall Street and the Tea Party, major newspapers' mentions of racism and white supremacy went through the roof. That's because, as Chomsky taught us, newspapers aren't there to inform you. They are there to manufacture consent, and one of the great ways to manufacture consent for the establishment is to divide and conquer. That's what identity politics is."
594
19,430
870
51,743
2,205,105
Marc Garcia retweeted
This meme was a small moment of enlightenment for me a long long while back
187
1,065
154
19,324
1,608,765
Marc Garcia retweeted
One day we are going to wake up, after years of the consistent and pernicious pollution of critical information ecosystems, and deeply regret this era of thoughtless LLM deployment. A devastating loss of accuracy, clarity and conciseness -- for what?
The internet has been progressively diluted with AI-generated slop. Are medical records headed for the same fate? 🧵 I just published a perspective in @NEJM with @AdamRodmanMD and Arjun Manrai on why rushing AI into medical documentation could be a mistake.
28
687
23
2,499
146,352
Marc Garcia retweeted
Scikit-Learn just crossed 100000 citations! I am no longer part of the core team, but I am proud to have been part of the project since the beginning. A big thank you to all maintainers, contributors, and users for their continuous vision, support and help! 👏 @scikit_learn
3
27
4
458
35,632
This video is the reason why I pay my internet bills for 😂🤣
973
10,213
1,119
71,350
5,202,729