Skip to content

Complete Guide to Getting Data from X (formerly Twitter) — February 2026

Executive Summary

X has undergone massive API restructuring since the Musk acquisition. The old tiered subscription model (Free/Basic/Pro/Enterprise) has been replaced with pay-per-usage pricing (no monthly subscriptions). The platform now offers modern v2 endpoints, a new X Activity API (XAA) for real-time events, a News API powered by Grok, Community Notes API, and webhooks. Scraping is harder than ever but not impossible. Here's every method available today.

1. Official X API — Current Structure

Pricing Model (NEW — Pay-Per-Usage)

X has moved away from the old Free/Basic ($100/mo)/Pro ($5,000/mo) subscription tiers to a pay-per-usage model. You pay only for what you use — no monthly caps or subscriptions.

  • Access: Sign up at console.x.com to manage apps, view usage, and purchase credits
  • Status: ✅ Working
  • Authentication: OAuth 1.0a (user context) and OAuth 2.0 (app-only / PKCE)

Available Endpoint Categories:

Category What You Get
Posts Create, read, search, delete posts; full-archive search back to 2006
Users Profile lookup, follower/following lists
Direct Messages Read/send DMs (with user auth)
Likes Get liked posts, liking users, like/unlike
Lists List management and membership
Spaces Lookup, search, ticket buyers
Communities Community data
Community Notes Search eligible posts, create/evaluate notes (pilot)
Trends Trending topics
News Grok-curated news stories and headlines (NEW)
Media Upload images, videos, GIFs
X Activity (XAA) Real-time event subscriptions (NEW)
Filtered Stream Real-time post delivery with up to 1,000 filter rules
Compliance Compliance streams
Webhooks Filtered stream & account activity via webhook delivery
Usage Monitor your API consumption

Key Features:

  • Full-archive search: Search ALL public posts back to 2006 via GET /2/tweets/search/all
  • Filtered stream: Up to 1,000 rules for real-time matching
  • Rich data objects: Posts (full text, metrics, entities, annotations, threads), Users (profiles, follower counts, verification), Media, Polls
  • Fields & Expansions: Customize responses to get exactly the data you need

Cost: Pay-per-usage (specific per-request pricing not publicly listed — check console.x.com) Gotchas: Must purchase credits upfront; pricing can be opaque

2. X API v2 vs v1.1

Version Status Notes
v2 ✅ Current — all new features Modern endpoints, flexible pricing, recommended
v1.1 ⚠️ Deprecated/Limited Legacy; some endpoints may still respond but not maintained
Enterprise (GNIP 2.0) ✅ Available High-volume access, dedicated support

Recommendation: Use v2 exclusively. v1.1 is on borrowed time.

3. Academic Research Access

  • Status: ❌ Discontinued
  • The Academic Research product track was eliminated in 2023 when Elon Musk restructured the API.
  • No free/discounted academic tier exists anymore.
  • Researchers must use the same pay-per-usage API as everyone else.
  • Some universities have institutional enterprise agreements — check with your library/IT.

Full-Archive Search (v2)

  • Status: ✅ Available via pay-per-usage
  • Endpoint: GET /2/tweets/search/all
  • Coverage: All public posts since 2006
  • Cost: Pay-per-usage (expensive at scale)

Enterprise Firehose (GNIP 2.0)

  • Status: ✅ Available (enterprise contracts only)
  • Products:
  • PowerTrack API: Complete real-time stream of all public posts, filterable
  • Decahose API: Random 10% sample of all public posts + likes
  • Historical PowerTrack: Batch access to full archive
  • Search API: 30-day and full-archive search
  • Compliance Firehose: All compliance events
  • Engagement API: Impressions and engagement metrics
  • Cost: Enterprise pricing — contact X sales (typically $10K–$100K+/month)
  • Access: Managed via Gnip Console, dedicated account managers

5. Third-Party Data Providers

Companies that have enterprise X API access and resell/provide data:

Provider What They Offer Status
Brandwatch (Cision) Social listening, historical data, sentiment analysis ✅ Active
Sprout Social Social media management, analytics, listening ✅ Active
Meltwater Media monitoring, social listening ✅ Active
Talkwalker (Hootsuite) Social listening, analytics ✅ Active
Sprinklr Enterprise social suite, listening ✅ Active
Synthesio (Ipsos) Social intelligence platform ✅ Active
NewsWhip Social media content prediction/analytics ✅ Active
Pulsar Audience intelligence, social listening ✅ Active
Socialbakers (Emplifi) Social media analytics ✅ Active

Cost: $500–$50,000+/month depending on scale Gotchas: Many had to renegotiate after X's API changes; data coverage may have gaps. Some may have lost firehose access. Always verify current X data coverage before signing.

6. Scraping Tools

Nitter

  • Status: ⚠️ Alive but requires real Twitter accounts
  • Now requires session tokens from real X accounts (guest API was killed)
  • Public instances are unreliable; most shut down
  • Self-hosting possible but you need accounts to feed it
  • Still provides RSS feeds, no-JS browsing
  • Risk: Account bans, session expiry

snscrape

  • Status: ⚠️ Largely broken for Twitter
  • GitHub repo still exists, lists Twitter support
  • In practice, X's anti-scraping measures have broken most Twitter functionality
  • May work intermittently; requires ongoing maintenance
  • Still works for other platforms (Instagram, Reddit, etc.)

Twint

  • Status: ❌ Dead
  • Unmaintained, completely broken since Twitter API changes in 2023

Other Scrapers

Tool Status
Twikit ⚠️ Intermittent — uses unofficial API, needs account auth
twitter-scraper (Python) ❌ Mostly broken
Selenium/manual scrapers ⚠️ Possible but fragile (see browser automation below)
yt-dlp ✅ Can download Twitter/X videos

Overall scraping verdict: Extremely difficult. X has aggressive bot detection, rate limiting, and legal enforcement. Any scraping approach requires authenticated accounts and is fragile.

7. Browser Automation (Playwright/Puppeteer)

  • Status: ⚠️ Possible but extremely fragile
  • X requires login for most content viewing
  • Heavy anti-bot detection (fingerprinting, CAPTCHAs, behavioral analysis)
  • Approaches:
  • Login with real account via Playwright
  • Use stealth plugins (playwright-extra, puppeteer-extra-plugin-stealth)
  • Rotate proxies, randomize behavior
  • Data accessible: Anything you can see as a logged-in user
  • Rate: Very slow compared to API
  • Risk: Account bans, IP blocks, ToS violation
  • Cost: Proxy costs ($50-500/mo for residential proxies)

Recommendation: Last resort. Use the API instead.

8. RSS Feeds

Native X RSS

  • Status: ❌ X killed native RSS feeds years ago

Nitter RSS

  • Status: ⚠️ Works on self-hosted instances (with account tokens)
  • Format: https://your-nitter-instance/username/rss
  • Provides user timelines, searches, hashtags as RSS
  • Requires maintaining Nitter + session tokens

Third-party RSS bridges

  • RSSHub (rsshub.app): ⚠️ X routes may be broken/limited
  • RSS.app, Feedbin: Some offer X-to-RSS conversion (paid services)
  • OpenRSS: ⚠️ Limited X support

Recommendation: Self-hosted Nitter is the most reliable RSS option, but requires maintenance.

9. Google Cache / Wayback Machine / Archives

Google Cache

  • Status: ❌ Google removed the cache feature in 2024
  • Google no longer provides cached page views

Wayback Machine (web.archive.org)

  • Status: ✅ Working for historical tweets
  • Many tweets were archived before they were deleted
  • Search: https://web.archive.org/web/*/twitter.com/username/status/*
  • Limited — only pages that were crawled
  • Cost: Free

Other Archives

  • Archive.today (archive.ph): ✅ Works, users can save snapshots of tweets
  • Politwoops (by ProPublica): ✅ Tracks deleted politician tweets
  • Thread Reader App (threadreaderapp.com): ✅ Archives tweet threads
  • Community Notes data dump: ✅ Available on GitHub (twitter/communitynotes)

10. X Ads API

  • Status: ✅ Working
  • Access: Requires approved Ads API access + active ad account
  • What you can do:
  • Campaign management (create, schedule, manage)
  • Custom/Tailored Audiences (upload your own data)
  • Creatives management (draft posts, cards, media)
  • Advertising analytics — granular campaign insights, impressions, engagements, conversions
  • Data available: Campaign performance metrics, audience insights, spend data
  • Cost: Requires active ad spend + API access approval
  • Tools: Postman collection available, xurl CLI tool

Gotcha: This is advertising data, not general public tweet data. Useful for marketers, not researchers.

11. X Analytics / Creator Studio

  • Status: ✅ Available to account owners
  • Access: analytics.x.com (for your own account)
  • Data available:
  • Tweet impressions, engagement rate, link clicks
  • Follower demographics (interests, location, gender)
  • Top tweets, mentions
  • Video views, completion rates
  • Monthly summary
  • Export: CSV export available for tweet-level data
  • Cost: Free (for your own account data)
  • Gotcha: Only YOUR account's data. No access to other accounts' analytics.

X Premium / X Blue Analytics

  • Premium subscribers get enhanced analytics including longer history

12. Grok on X

  • Status: ✅ Active and integrated into X
  • What Grok can access:
  • Real-time X posts and conversations
  • Trending topics and discussions
  • Can search and summarize X content on any topic
  • Powers the News API (Grok-curated trending stories)
  • News API: GET /2/news/{id} and GET /2/news/search — returns Grok-curated news with summaries, context entities, related post IDs
  • How to use for research:
  • Ask Grok questions about what's being discussed on X
  • Grok can analyze sentiment, summarize threads, identify trends
  • Available via X Premium subscription or xAI API
  • Limitations: Grok summaries may be inaccurate; limited to what Grok can see; not a structured data export tool
  • Cost: X Premium subscription ($8-16/mo) for in-app Grok; xAI API for programmatic access

13. Community Notes API

  • Status: ✅ Working (Pilot)
  • Endpoints:
  • GET /2/notes/search/posts_eligible_for_notes — find posts eligible for notes
  • GET /2/notes/search/notes_written — get notes you've written
  • POST /2/notes — create a community note
  • DELETE /2/notes/{id} — delete a note
  • POST /2/evaluate_note — evaluate before submitting
  • Requirements: Must be enrolled as an AI Note Writer in Community Notes
  • Rate limits: 90 requests/15 min; 250 creates/day
  • Cost: Pay-per-usage API pricing

Community Notes Open Data

  • Status: ✅ Available
  • All Community Notes data is published openly at communitynotes.x.com
  • Downloadable TSV files with notes, ratings, and note status history
  • Cost: Free
  • This is the best source for Community Notes research

14. X Spaces

  • Status: ✅ API available
  • Endpoints:
  • Lookup by Space ID (single or batch)
  • Lookup by creator ID
  • Search Spaces by keyword
  • Get ticket buyers list
  • Data available: Space metadata (title, state, host, speakers, participant count, scheduled start, language)
  • Limitations:
  • No audio streaming/recording via API
  • Spaces are ephemeral — unavailable after ending
  • Only live and scheduled Spaces are accessible
  • Recording Spaces: No official API. Third-party tools and browser extensions exist (risky, ToS violation)
  • X Activity API: Can subscribe to spaces.start and spaces.end events for specific users

15. Webhooks / Account Activity API

Webhooks (v2)

  • Status: ✅ Working
  • Types:
  • Filtered Stream Webhooks: Receive matching posts via webhook instead of persistent connection
  • Account Activity API: Receive events for subscribed accounts
  • Requirements: HTTPS endpoint, CRC validation, respond within 10 seconds
  • Security: HMAC-SHA256 signature verification

X Activity API (XAA) — NEW

  • Status: ✅ Working
  • A major new addition — real-time event subscriptions
  • Delivery: Persistent HTTP stream OR webhooks
  • Event types:
  • Profile updates (bio, picture, banner, username, location, URL, badges)
  • Follow/unfollow events
  • Spaces start/end
  • DM sent/received (private — requires OAuth)
  • News events (Grok-curated trends)
  • Subscription limits:
  • Self-serve: 1,000 subscriptions
  • Enterprise: 50,000
  • Partner: 100,000
  • Latency: Sub-second

Legacy Account Activity API (Enterprise/GNIP)

  • Status: ✅ Available via enterprise
  • Posts, DMs, likes, follows, and more

Terms of Service

  • Scraping is explicitly prohibited by X's ToS
  • Automated access without API authorization is prohibited
  • X has sued scrapers (e.g., multiple lawsuits in 2023-2024)
  • X sends cease-and-desist letters to scraping projects

Rate Limits

  • API: Per-endpoint rate limits (varies); usage-based billing
  • Aggressive rate limiting on unauthenticated access
  • IP-based rate limiting on x.com itself

GDPR / Privacy

  • X data may contain personal data subject to GDPR
  • Redistribution of X data has restrictions
  • Must comply with X's Developer Agreement

IP Bans

  • X actively blocks datacenter IPs, VPN ranges, and known scraping IPs
  • Residential proxies are harder to block but still detected via behavioral analysis

Research Exceptions

  • No formal academic exemption exists anymore
  • Some researchers argue fair use; untested in most jurisdictions for API scraping

17. Open Source Alternatives

Bluesky (AT Protocol)

  • Status: ✅ Fully open, growing rapidly
  • API: Free, open, no authentication required for public data
  • Public API: https://public.api.bsky.app
  • Endpoints: Profiles, posts, feeds, search, notifications, moderation, lists
  • Firehose: Available — can subscribe to all public events
  • Rate limits: Generous
  • Cost: Free
  • Libraries: Python (atproto), TypeScript (@atproto/api), etc.
  • Best for: Researchers who need open social media data

Mastodon (ActivityPub/Fediverse)

  • Status: ✅ Open, federated
  • API: RESTful, well-documented, free
  • Data: Public posts, profiles, trends per instance
  • Limitations: Federated nature means no single firehose; each instance has its own data
  • Cost: Free

Threads (Meta)

  • Status: ⚠️ Limited API (via Instagram Graph API)
  • Less open than Bluesky

Recommendation: If you need open social media data for research, Bluesky is the best alternative — fully open protocol, free firehose, no authentication barriers for public data.

18. Data Marketplaces

Source What's Available Status
Kaggle Historical Twitter datasets (pre-2023 mostly) ✅ Free
Archive.org Twitter Stream archives (historical) ✅ Free
Pushshift Limited Twitter data (Reddit-focused) ⚠️ Limited
Bright Data Web scraping infrastructure + datasets ✅ Paid
Data.world Some Twitter datasets ✅ Mixed
GDELT Tracks URLs shared on Twitter ✅ Free
Docnow Catalog Tweet ID datasets for research ✅ Free (IDs only — must hydrate)

Gotcha: Most historical datasets contain only tweet IDs (not full text) due to ToS. You must "hydrate" them via the API, but deleted tweets won't hydrate.

19. OSINT Tools

Tool What It Does Status
Social Searcher Social media search engine ✅ Working (limited X data)
Twint OSINT Twitter scraper ❌ Dead
Maltego Link analysis with Twitter transforms ⚠️ Limited (API-dependent)
SpiderFoot OSINT automation ⚠️ Limited X modules
Sherlock Username search across platforms ✅ Works for checking X usernames
Maigret Advanced username search ✅ Works
TweetBeaver Twitter analytics/OSINT ⚠️ Unreliable
Followerwonk (Moz) Twitter analytics ⚠️ Limited after API changes
Botometer (Indiana U) Bot detection ⚠️ Depends on API access
Bellingcat tools Various OSINT ⚠️ Twitter-specific tools degraded

Verdict: Most dedicated Twitter OSINT tools are degraded or broken. The API is now the primary path for any reliable OSINT work on X.

20. Embedded Tweets / oEmbed

  • Status: ✅ Working
  • oEmbed API: Returns embed HTML for any public tweet
  • Endpoint: https://publish.twitter.com/oembed?url=https://x.com/user/status/123
  • Embedded Tweets: Copy/paste HTML or use oEmbed API
  • Embedded Timelines: Profile or list timeline embeds
  • X Publish: publish.x.com — visual embed builder
  • Tweet/Follow Buttons: Still available
  • WordPress plugin: Available
  • Cost: Free
  • Gotcha: Embedded tweets load via JavaScript from X's servers; they break if the tweet is deleted

🏆 Recommendations: What's Actually Worth Using Today

For Developers / Businesses

  1. X API v2 (pay-per-usage) — The primary legitimate path. Full-archive search, streaming, all data types.
  2. X Activity API (XAA) — Excellent for real-time monitoring of specific accounts/events.
  3. Webhooks — Better than polling for event-driven architectures.

For Researchers

  1. X API v2 — Pay-per-usage; budget carefully.
  2. Community Notes open data — Free, comprehensive, downloadable.
  3. Bluesky AT Protocol — Free, open alternative with growing user base.
  4. Wayback Machine / Archive.today — For historical content.

For Social Listening / Marketing

  1. Third-party providers (Brandwatch, Meltwater, Sprinklr) — Easier than direct API, bundled with analytics.
  2. X Ads API — If you're running ads anyway.
  3. Grok / News API — For trending topic analysis.

For OSINT / Investigations

  1. X API v2 — Legitimate, reliable.
  2. Nitter (self-hosted) — For privacy-preserving browsing + RSS.
  3. Archive services — Wayback Machine, archive.today.
  4. Community Notes data — Publicly available.

❌ Avoid

  • snscrape / Twint — Broken or unreliable
  • Unauthenticated scraping — Will get you blocked immediately
  • Browser automation at scale — Fragile, expensive, risky

💡 Sleeper Pick

  • Bluesky's open API + firehose — If your use case can work with Bluesky data instead of X data, it's dramatically easier, cheaper, and more reliable. Growing user base makes it increasingly viable.

Report compiled February 12, 2026. X's API landscape changes frequently — verify current pricing and availability at docs.x.com and console.x.com.