What this project does

Most leagues are decided on draft day, and the tools I was using were either paywalled or too generic to help with the picks that mattered. I wanted one place that covered the whole season, from the draft board to the Tuesday waiver claim.

Building the data

The model pipeline runs as three notebook phases. The first builds one dataset per position from nflverse, the second engineers features and the third trains the models. Each phase saves its output for the next one, and the final predictions feed the rankings page and the mock draft.

What the model predicts

The target is a player's points per game in the next season, so every row is one player-season with that following season's PPG attached. This is the function that builds it.

def create_target(df, id_col='player_id', season_col='season', ppg_col='ppg'):
    """
    Add next_ppg = the player's PPG in their next season.
    Rows where next_ppg is NaN = the player had no following season in the dataset.
    The 2025 feature rows will have next_ppg=NaN — these are what we predict for 2026.
    """
    # M1 FIX: join on the stable gsis id, not the display name — nflverse
    # renames players between seasons (Chris Godwin -> Chris Godwin Jr.,
    # DJ Moore, AJ Dillon, ...) and 24 player-seasons lost their lag
    # features to it. Falls back to name only if the id column is absent.
    if id_col == 'player_id' and id_col not in df.columns:
        id_col = 'player_name'
    df = df.sort_values([id_col, season_col]).copy()
    df['next_ppg'] = df.groupby(id_col)[ppg_col].shift(-1)
    return df

The comment on the join is a bug I fixed. I originally joined seasons on player name, but nflverse renames players between seasons, like Chris Godwin to Chris Godwin Jr. That quietly dropped the lag features for 24 player-seasons. Joining on the stable player ID fixed it.

Two models per position

Each position gets a Ridge regression and an XGBoost model. Ridge is a linear baseline with median imputation and standardization. XGBoost is capped at a shallow depth so it can't overfit the small per-position datasets.

def make_ridge():
    """Ridge pipeline: median impute NaN, standardize, then Ridge."""
    return Pipeline([
        ('imputer', SimpleImputer(strategy='median')),
        ('scaler',  StandardScaler()),
        ('model',   Ridge(alpha=10.0))
    ])

def make_xgb():
    """XGBoost — handles NaN natively, capped depth to avoid overfitting on small datasets."""
    return xgb.XGBRegressor(
        n_estimators=200,
        max_depth=3,
        learning_rate=0.05,
        subsample=0.8,
        colsample_bytree=0.7,
        min_child_weight=5,
        reg_alpha=0.1,
        reg_lambda=1.0,
        random_state=42,
        verbosity=0
    )

Testing without cheating

Random train/test splits leak the future in sports data, so I use walk-forward validation. To test on 2025, I train only on seasons before 2024 and use 2024 stats to predict 2025. Then I compare Ridge, XGBoost and a 50/50 blend on mean absolute error.

If one model wins by less than 5% over the blend, I use the blend, because a small edge on a small test set isn't worth the instability.

def pick_strategy(cv_results):
    """
    Compare average MAE across CV folds.
    If difference between winner and blend is < 5%, use blend (safer).
    Otherwise use the clear winner.
    """
    if not cv_results:
        return 'blend'

    avg = {'ridge': 0, 'xgb': 0, 'blend': 0}
    for r in cv_results.values():
        avg['ridge'] += r['ridge_mae']
        avg['xgb']   += r['xgb_mae']
        avg['blend']  += r['blend_mae']

    n = len(cv_results)
    avg = {k: v / n for k, v in avg.items()}
    best = min(avg, key=avg.get)
    second = sorted(avg, key=avg.get)[1]

    # If best is within 5% of blend, prefer blend for stability
    if best != 'blend' and abs(avg[best] - avg['blend']) / avg['blend'] < 0.05:
        return 'blend'
    return best

The final predictions also lean on a player's history. For players with fewer than three seasons of data, I blend the model's prediction toward their recent weighted PPG, so a rookie with two good games doesn't get ranked like a veteran.

confidence     = np.minimum(seasons_played / 3.0, 1.0)
raw_preds      = np.maximum(strategy_preds, 0)
anchored_preds = confidence * raw_preds + (1 - confidence) * pred['weighted_ppg'].values

Finding similar players

A projection alone doesn't explain why a player is ranked where he is. For each player entering 2026, I look for the five most similar player-seasons from 2021 on, using age, experience, last season's production and usage, and whether he changed teams. The comps are shown on each player's page next to what those players did the following year.

scaler      = StandardScaler()
hist_scaled = scaler.fit_transform(hist[feat_cols])
curr_scaled = scaler.transform(curr[feat_cols])

nn = NearestNeighbors(n_neighbors=K_COMPS, metric='euclidean')
nn.fit(hist_scaled)
distances, indices = nn.kneighbors(curr_scaled)

Backtesting the draft strategy

Accurate projections only matter if they make better drafts. So I simulate them. For the 2025 season, I ran 10,000 snake drafts in a 12-team PPR league. One strategy drafts from the model's walk-forward predictions, the other drafts from a consensus proxy built on past PPG, and I score both using what actually happened in 2025.

NUM_SIMS        = 10_000
NUM_TEAMS       = 12
ROUNDS          = 15
GAMES_IN_SEASON = 17
ADP_NOISE_STD   = 8.0

The Draft AI

On draft day, the best player available is not always the best pick. First, the Draft AI measures how many startable players are left at each position and labels each one from DEEP to CRITICAL.

critical_threshold = max(2, num_teams // 3)
tight_threshold = max(4, num_teams // 2)

scarcity[pos] = {
    "total_available": len(available_at_pos),
    "above_replacement": n_above,
    "top_3": available_at_pos[:3],
    "scarcity_level": (
        "CRITICAL" if n_above <= critical_threshold
        else "TIGHT" if n_above <= tight_threshold
        else "MODERATE" if n_above <= 10
        else "DEEP"

Then it scores the best 30 players on the board by value and by how much your roster needs the position. The weight on need grows as the draft goes on, so early rounds lean on value and late rounds fill holes.

if current_round <= 2:
    need_w = 3.0
elif current_round <= 5:
    need_w = 8.0
elif current_round <= 9:
    need_w = 14.0
else:
    need_w = 22.0

best, best_score, best_reason = None, None, ""
for idx, p in enumerate(pool[:30]):
    pos = str(_pkey(p, "position", "Position")).upper().replace("DEF", "DST")
    if pos in ("K", "DST") and current_round < 11:
        continue
    value_score = 30 - idx
    gap = gap_by_pos.get(pos, 0)
    need_score = min(gap, 2) * 1.0
    scarce = ""
    if scarcity and pos in scarcity:
        level = scarcity[pos].get("scarcity_level")
        if gap > 0 and level in ("CRITICAL", "TIGHT"):
            need_score += 0.75
            scarce = f", {pos} is running out"
    score = value_score + need_w * need_score
    if best_score is None or score > best_score:
        best, best_score = p, score
        if gap > 0:
            best_reason = f"Fills your open {pos} starter slot{scarce}"
        else:
            best_reason = "Best available for your roster"

Live draft sync

Drafts update live. One greenlet per active draft polls Sleeper every three seconds and writes new picks to Supabase, and Supabase Realtime pushes them to everyone watching the board.

"""DraftPollerManager — server-side poller for live draft sync via Supabase Realtime.

One greenlet per active draft polls Sleeper/ESPN every 3 seconds, writes new picks
to Supabase (triggering Realtime push to all subscribed clients).
"""

This is also where my worst production bug lived. In week 6, I found four pollers for finished drafts still running, over 33,000 Sleeper requests since boot. Each resumed poller kept refreshing its own last_polled_at, so the staleness check could never retire it. The fix was to age sessions on scheduled_start, which is written once, instead of a timestamp the poller itself keeps updating.

League Hub

Once the draft is over, the site shifts to the season. League Hub connects a Sleeper or ESPN league and brings the same projections to every weekly decision. Two league platforms means two different data shapes, so I normalize both into one internal format, and everything below runs on top of it.

Rest-of-season rankings

Preseason projections get stale quickly. Each week, I rebuild a rest-of-season ranking that blends the model's projection with what a player has actually done. The trust I put in actual results grows with games played, so one quiet opener doesn't crater a star, and a hot three-game stretch doesn't make a backup look like a starter.

# R1: actual-vs-model trust follows GAMES PLAYED, not the calendar week:
#   actual_weight = n / (n + ROS_SHRINKAGE_K)
# n=1 → 0.10, n=3 → 0.25, n=9 → 0.50, n=15 → 0.625 — matches the old
# mid-season ladder while making a 1-game sample gentle automatically
# (the old week-keyed 25% cratered stars off one quiet opener).
ROS_SHRINKAGE_K = 9

For running backs, receivers and tight ends, the actual side is anchored to what their usage says they should score, meaning targets and carries times league rates, so touchdown luck counts for less. Players with no games in a started season get a mild availability discount, and players whose season is over are dropped by detecting season-ending injuries from the provider and the news.

Start/Sit

Start/Sit is the tool I use the most myself. For every player on your roster, it builds a projection rating, a floor, a ceiling and a matchup rating, then combines them into one start score. You can weight those for a safe or a risky lineup, and an optimizer fills your slots. It reuses the same data sources as the rankings and waiver tool, so every number on the page comes from the same place.

RISK_PROFILES = {
    "safe":     {"projection": 0.61, "floor": 0.30, "ceiling": 0.05, "matchup": 0.04},
    "balanced": {"projection": 0.61, "floor": 0.175, "ceiling": 0.175, "matchup": 0.04},
    "upside":   {"projection": 0.61, "floor": 0.05, "ceiling": 0.30, "matchup": 0.04},
}
START_SIT_WEIGHTS = RISK_PROFILES["balanced"]

If a rating is missing for a player, the score doesn't fall apart. The missing component is dropped and the remaining weights are rescaled, so one gap in the data lowers confidence instead of zeroing the score.

def calculate_start_score(projection, floor, ceiling, matchup, weights=None):
    """Weighted Start Score (0-100) from the four component ratings.

    Any component that is None (data unavailable) is excluded and the
    remaining weights are renormalized proportionally, so a single missing
    signal degrades the score gracefully instead of zeroing it out or
    requiring a fabricated placeholder value. Returns None only when every
    component is None.
    """
    weights = weights or START_SIT_WEIGHTS
    parts = {"projection": projection, "floor": floor, "ceiling": ceiling, "matchup": matchup}
    available = {k: v for k, v in parts.items() if v is not None}
    if not available:
        return None
    total_weight = sum(weights.get(k, 0.0) for k in available) or 1.0
    score = sum(available[k] * weights.get(k, 0.0) for k in available) / total_weight
    return round(score, 1)

Checking my own accuracy

Ratings are easy to claim and hard to prove. So every Start/Sit rating is saved before that week's games are played, and graded once the week is final. A number computed after the fact can always be cherry-picked, so the order is the whole point: freeze first, grade later.

Trade Analyzer

The trade analyzer takes both rosters and grades a trade by how it changes your lineup and your depth. Lineup changes count for 75% and bench depth for 25%. The engine is pure functions with no Flask and no database, which makes it easy to test and lets the UI and the narration read from the same verdict.

SLOT_ELIGIBILITY = {
    'QB':  {'QB'}, 'RB': {'RB'}, 'WR': {'WR'}, 'TE': {'TE'},
    'K':   {'K'},  'DST': {'DST'}, 'DEF': {'DST'},
    'WRRB_FLEX':  {'WR', 'RB'},
    'REC_FLEX':   {'WR', 'TE'},
    'FLEX':       {'RB', 'WR', 'TE'},
    'SUPER_FLEX': {'QB', 'RB', 'WR', 'TE'},
}
# Any slot label not in the map (IDP_FLEX, LB, DB, DL, ...) is unfillable:
# it stays in the lineup as an empty slot and evaluate_trade flags it once.

# Suggestion utility: starter PPG counts more than bench PPG above replacement.
# Letter grades use valuation.value_balance, on a separate percent scale.
GRADE_LINEUP_WEIGHT = 0.75
GRADE_DEPTH_WEIGHT = 0.25

Flex slots were the hard part. Filling them greedily one at a time is wrong when slots overlap, because taking a receiver into one flex can leave another empty when a swap would have filled both. Flex counts are small, so the engine searches every assignment and takes the best one.

Waivers and FAAB

Waiver recommendations use the same ratings, filtered to players who are actually available and fit your roster. For leagues with a budget, the site also suggests a FAAB bid, calibrated against recent winning bids in that league. Getting the league type right mattered more than I expected, because a positive budget alone doesn't mean a league uses FAAB.

def league_uses_faab(settings: LeagueSettings) -> bool:
    """True only when the league actually uses acquisition-budget waivers.

    Sleeper (verified via live leagues + transaction ``waiver_bid`` fields):
      0 = rolling waivers, 1 = reverse-standings priority, 2 = FAAB.
    A positive ``waiver_budget`` alone does *not* mean FAAB.

Keeping it fast and reliable

League pages combine a lot of data, so I cache them in two tiers. Memory comes first, and a Postgres table sits behind it so the cache survives restarts and is shared across workers.

"""Hub payload cache port (L1 memory + optional L2 Postgres).

``HUB_CACHE_BACKEND``:
  memory — per-process dict only (default).
  dual / supabase — memory L1 + SQLAlchemy ``hub_cache_entries`` L2.
"""

The Sleeper player list is about 5MB, so no user request should ever wait on it. If the cached copy has expired, the site serves the stale rows right away and refreshes in a background thread.

if has_ranks:
    # Expired but usable: serve the stale rows now and refresh in the
    # background so no user request ever waits on the ~5MB Sleeper
    # player dump + full-table upsert.
    self._refresh_in_background(client)
    return self._cached_players(include_expired=True)

The week turns over on Tuesday. After the rest-of-season rebuild around 6 AM ET, a scheduled job warms every league's payloads and generates Start/Sit notes, with a deadline in case nflverse publishes late. PostHog shows me which features people use, and pytest with CI runs on every push so a change to the models can't quietly break the site.

What this project does

When you book a flight, the only delay information you get is whether the airline calls it on time. I wanted something more useful: for this airline, at this airport, in this month, how late should I expect to be?

Loading and shaping the data

The raw file has a lot of columns I don't need. I keep five: the airline code, the origin airport, the flight date, the departure delay in minutes and whether the flight was cancelled. Then I pull the month out of the date, because the month is what captures seasonality.

Airline, airport and month are all categories, so I one-hot encode them. I hold out 20% of the flights for testing.

import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestRegressor

df = pd.read_csv('CX4242_FlightData.csv')

df = df.iloc[:, [0, 1, 2, 6, 10]]

df['FLIGHT_DATE'] = pd.to_datetime(df['FLIGHT_DATE'])
df['Month_of_Travel'] = df['FLIGHT_DATE'].dt.month
X = df[['AIRLINE_CODE', 'ORIGIN_CODE', 'Month_of_Travel']]
y_delay = df['DEP_DELAY']
y_cancel = df['CANCELLED']

X_encoded = pd.get_dummies(X)

X_train, X_test, y_delay_train, y_delay_test, y_cancel_train, y_cancel_test = train_test_split(
    X_encoded, y_delay, y_cancel, test_size=0.2, random_state=42)

Training the delay model

I fit a RandomForestRegressor on the encoded features to predict departure delay in minutes. I picked a random forest because the effect of an airline depends on the airport and the month, and a forest picks up those interactions without me having to specify them.

delay_model = RandomForestRegressor().fit(X_train, y_delay_train)

df['Predicted_Delay_Time'] = delay_model.predict(X_encoded)

Turning predictions into a lookup table

The tool needs to answer instantly, so I don't run a model in the browser. Instead I average the predicted delay for every airline, airport and month, floor it at zero so nobody is promised an early departure, and take the share of cancelled flights for each group. That produces flightdata_delay_and_cancellation.csv, which the front end reads directly.

grouped_df = df.groupby(['AIRLINE_CODE', 'ORIGIN_CODE', 'Month_of_Travel']).agg({
    'Predicted_Delay_Time': lambda x: max(x.mean(), 0),
    'CANCELLED': 'mean'
}).reset_index()

grouped_df = grouped_df.rename(columns={'CANCELLED': 'Cancellation_Probability'})

grouped_df.to_csv('flightdata_delay_and_cancellation.csv', index=False)

Checking it against other models

To see whether a Random Forest was the right choice, I turned the same problem into a yes or no question: is a flight more than 15 minutes late? I trained a Random Forest classifier and a Gradient Boosting classifier on the same three features and compared accuracy on the held-out flights.

X = df.iloc[:, [0,1,2]]
y = df["DEP_DELAY"].apply(lambda x: 0 if x <= 15 else 1) 

y = y.fillna(y.median()).astype(float)

encoder = OneHotEncoder()
X_encoded = encoder.fit_transform(X)

X_train, X_test, y_train, y_test = train_test_split(X_encoded, y, test_size=0.2, random_state=42)

clf = GradientBoostingClassifier(random_state=42)

clf.fit(X_train, y_train)

Gradient Boosting also cut forecast error by 37% against a basic baseline when I tested it on held-out flight data.

The tool

Pick an airline, an airport and a month and the tool shows the expected delay and the chance of cancellation. Here it is on load, then after I pick Delta, Atlanta and July.

Exploring by airport

You can also hover any airport on the map to see the five airlines with the lowest expected delay there. That view is more useful when you are choosing an airline than when you already have one picked.

What this project does

Employees lose hours hunting through policy PDFs for one rule or limit. I built an assistant that turns that into a single question and shows which document the answer came from, so it can be checked.

Ingesting a document

When you upload a PDF, the API loads it, splits it into overlapping chunks, tags each chunk with the document it came from and stores the embeddings in Pinecone. Before writing, it deletes any vectors that already belong to that document, so re-uploading a policy replaces the old version instead of piling up beside it.

import { PDFLoader } from "@langchain/community/document_loaders/fs/pdf";
import { RecursiveCharacterTextSplitter } from "langchain/text_splitter";
import { OpenAIEmbeddings } from "@langchain/openai";
import { PineconeStore } from "@langchain/pinecone";
import { Pinecone } from "@pinecone-database/pinecone";
import * as dotenv from "dotenv";

dotenv.config();

interface IngestResult {
  chunksIngested: number;
  docId: string;
}

export async function ingestDocument(
  filePath: string,
  docId: string
): Promise<IngestResult> {
  // 1. Load PDF
  const loader = new PDFLoader(filePath, {parsedItemSeparator: " ",});
  const pages = await loader.load();

  // 2. Chunk — experiment with these values for your evals
  const splitter = new RecursiveCharacterTextSplitter({
    chunkSize: 300,
    chunkOverlap: 50,
    separators: ["\n\n", "\n", ".", " "],
  });
  const chunks = await splitter.splitDocuments(pages);

  // 3. Add metadata to each chunk
  chunks.forEach((chunk, i) => {
    chunk.metadata = {
      ...chunk.metadata,
      doc_id: docId,
      chunk_index: i,
    };
  });

  // 4. Embed and upsert to Pinecone
  const pinecone = new Pinecone({ apiKey: process.env.PINECONE_API_KEY! });
  const index = pinecone.Index(process.env.PINECONE_INDEX_NAME!);

  const embeddings = new OpenAIEmbeddings({
    model: "text-embedding-3-small",
  });

  // Delete existing vectors for this document before upserting new ones
  await index.deleteMany({ doc_id: docId });

  await PineconeStore.fromDocuments(chunks, embeddings, {
    pineconeIndex: index,
  });

  return { chunksIngested: chunks.length, docId };
}

Retrieval

The retriever is intentionally small. It opens the existing Pinecone index and returns the top K closest chunks to a question. K is a parameter because it ended up being one of the settings I tested.

import { OpenAIEmbeddings } from "@langchain/openai";
import { PineconeStore } from "@langchain/pinecone";
import { Pinecone } from "@pinecone-database/pinecone";
import * as dotenv from "dotenv";

dotenv.config();

export async function getRetriever(topK: number = 10) {
  const pinecone = new Pinecone({ apiKey: process.env.PINECONE_API_KEY! });
  const index = pinecone.Index(process.env.PINECONE_INDEX_NAME!);

  const embeddings = new OpenAIEmbeddings({
    model: "text-embedding-3-small",
  });

  const vectorStore = await PineconeStore.fromExistingIndex(embeddings, {
    pineconeIndex: index,
  });

  return vectorStore.asRetriever({ k: topK });
}

Answering with a source

The answer step retrieves the chunks, puts them into a prompt and asks GPT-4o-mini to answer from that context only. If the answer isn't in the context, it has to say so instead of guessing. I fetch the chunks separately from the chain so I can return the document names next to the answer.

import { ChatOpenAI } from "@langchain/openai";
import { PromptTemplate } from "@langchain/core/prompts";
import { StringOutputParser } from "@langchain/core/output_parsers";
import { RunnablePassthrough, RunnableSequence } from "@langchain/core/runnables";
import { Document } from "@langchain/core/documents";
import { getRetriever } from "./retriever";

const PROMPT_TEMPLATE = `You are an enterprise document assistant. Use only the 
context below to answer the question. When comparing numerical values, explicitly 
state both numbers and the direction of comparison before drawing a conclusion. 
When answering, always cite the specific policy rule that applies, 
including any eligibility requirements or conditions, even if the 
user did not explicitly ask about them.
If the answer is not in the context, say 
"I don't have that information in the provided documents."

Context:
{context}

Question: {question}

Answer:`;

export interface QAResult {
  answer: string;
  sources: string[];
}

// Formats retrieved Document objects into a single context string for the prompt
function formatDocs(docs: Document[]): string {
  return docs.map((doc) => doc.pageContent).join("\n\n");
}

export async function queryDocuments(question: string): Promise<QAResult> {
  const llm = new ChatOpenAI({
    model: "gpt-4o-mini",
    temperature: 0,
  });

  const retriever = await getRetriever(4);
  const prompt = PromptTemplate.fromTemplate(PROMPT_TEMPLATE);

  // Retrieve documents separately so we can return sources alongside the answer
  const retrievedDocs = await retriever.invoke(question);

  // LCEL chain: format context + pass question → prompt → LLM → parse output
  const chain = RunnableSequence.from([
    {
      context: () => formatDocs(retrievedDocs),
      question: new RunnablePassthrough(),
    },
    prompt,
    llm,
    new StringOutputParser(),
  ]);

  const answer = await chain.invoke(question);

  const sources: string[] = [
    ...new Set(
      retrievedDocs.map(
        (doc) => doc.metadata?.doc_id ?? "unknown"
      )
    ),
  ];

  return { answer, sources };
}

Writing the test set

Before changing anything, I wrote 20 questions with known answers across three policy documents: expense, HR and IT security. They run from single facts to questions that need two sections of a policy at once. Each one records which document the answer lives in, which matters in the next step.

[
  {
    "id": 1,
    "question": "What is the reimbursement limit for a business meal per person?",
    "expectedAnswer": "$75 per person.",
    "sourceDocs": ["expense_policy.pdf"]
  },
  {
    "id": 3,
    "question": "Within how many hours of discovery must a security incident be reported?",
    "expectedAnswer": "Within 24 hours of discovery.",
    "sourceDocs": ["it_security_policy.pdf"]
  }
]

Telling retrieval failures from generation failures

A wrong answer can fail in two ways. Either the right chunk never came back, or it came back and the model still got it wrong. Those need different fixes, so the eval runner classifies every miss. A second model judges whether each answer matches the expected one, and then I check whether the expected document was among the retrieved sources.

// If wrong: was the source doc even retrieved? If not -> retrieval failure.
// If source doc was retrieved but still wrong -> generation failure.
let failureType: FailureType = "none";
if (!correct) {
  const sourceWasRetrieved = q.sourceDocs.every(
    (doc) => retrievedSourceDocs.includes(doc));
  failureType = sourceWasRetrieved ? "generation" : "retrieval";
}

What the experiments showed

I changed one thing at a time and re-ran all 20 questions. At baseline, most of the misses were generation failures. Adding an explicit instruction to state both numbers and the direction of comparison fixed most of them. Raising top-K to 10 got me to 95%. Smaller chunks and more overlap didn't beat it, so I settled on 300 / 50 / top-10.

One question still misses. It is a hard case that needs an eligibility condition from a different section of the policy.

What would break at scale

This works for a handful of documents, and I wrote down what wouldn't hold up past that, along with how I would fix each one.