Scoring Model Comparison: CollegeVine vs. Our Simulation
scoring_model_comparison.md
Scoring Model Comparison: CollegeVine vs. Our Simulation
Side-by-side analysis of CollegeVine's chancing engine (~75 factors, ML) vs. our agent-based simulation (logistic model, ~30 explicit factors). March 2026.
Table of Contents
- Architecture Overview
- Input Factors — Side by Side
- Academic Scoring
- Extracurriculars
- Essays & Recommendations
- Hooks & Demographic Factors
- Early Decision / Early Action
- Demonstrated Interest
- Application Strategy (List Building)
- Calibration & Accuracy
- What We Model That CollegeVine Can't
- What CollegeVine Models That We Don't
- Structural Differences
- Summary Scorecard
1. Architecture Overview
| Dimension | CollegeVine | Our Simulation |
|---|---|---|
| Model type | Machine learning (undisclosed algorithm; post-May 2021) | Explicit logistic model (all factors additive in logit space) |
| Training data | Self-reported outcomes from users + IPEDS + CDS | CDS data + SFFA trial data + Chetty 2023 + NBER papers |
| Output | P(admit) per student-college pair | P(admit) per student-college pair → Bernoulli trial |
| Transparency | Black box ("trade secrets") | Fully inspectable — every coefficient visible in sim.js |
| College coverage | 1,500+ colleges | 55 colleges (6 tiers, HYPSM through Selective Public) |
| Student representation | Individual user profiles | Agent archetypes (6 behavioral × 4 structural) from 20–485 high schools |
| Simulation scope | Single-student prediction | Full market simulation: 6 rounds, seat constraints, yield, waitlist, melt |
| Goal | "What are MY chances?" | "How does the admissions market work as a system?" |
The Fundamental Difference
CollegeVine answers: "Given your profile, what percentage of students like you get admitted to School X?"
Our simulation answers: "Given the full population of applicants, seat constraints, and round dynamics, what happens in the aggregate and to tracked individuals?"
CollegeVine is a prediction tool (frequentist probability for an individual). Our simulation is a market model (system dynamics with emergent outcomes).
2. Input Factors — Side by Side
Factors Both Models Use
| Factor | CollegeVine | Our Simulation | Notes |
|---|---|---|---|
| GPA (unweighted) | Direct input, compared to P25 of admits | gpa_uw → gpaToCSG() → Academic Index |
Both use UW GPA; we convert via a step-function lookup table |
| SAT/ACT | Direct input, compared to P25 of admits | sat + sat_math + sat_rw → section-weighted AI |
We also model section scores with per-college mathWeight |
| Course rigor (APs) | Number of AP/Honors courses | ap_count stored but not directly in scoring |
We generate AP counts per archetype but don't score them directly — course rigor is implicitly captured in GPA and archetype |
| Extracurriculars | 4-tier/12-sub-tier classification | ec_quality (1–10 continuous) |
See Section 4 |
| Gender | Input factor | Three gender models: stem_heavy, balanced, lac with explicit multipliers (Caltech 1.9× female, Williams 1.25× male) |
Both model this; ours is more transparent |
| State of residence | In-state vs. out-of-state for publics | student.state → rateInState/rateOOS logit adjustment + UNDERREP_STATE_BOOST for privates |
Both handle IS/OOS; we add geographic diversity for privates |
| Intended major | Grouped by "areas of study" | Explicit CS/Engineering/Humanities/Pre-Med penalties at specific colleges (CMU CS ×0.50, MIT CS ×0.50) | We model specific oversubscribed majors; CV groups broadly |
| Race/ethnicity | "Greatly reduced" post-SFFA | SFFA_MULT_BY_TIER — tier-differentiated (T1 URM 0.85×, Asian 1.35×; T5–T6 URM 1.05–1.08×) |
We model the post-SFFA countertrend at publics |
| ED/EA round | Added May 2022 | Full 6-round model: ED, EA, REA, EDII, RD, Waitlist with per-college edMultOverride |
See Section 7 |
Factors Only CollegeVine Uses
| Factor | CollegeVine | Why We Don't |
|---|---|---|
| Class rank | Direct input | Implicitly captured by GPA relative to high school's distribution |
| 75 claimed factors | Undisclosed full list | We use ~30 explicit factors — fewer but each one is transparent and calibrated |
Factors Only Our Simulation Uses
| Factor | Our Simulation | Why CollegeVine Doesn't |
|---|---|---|
| Legacy status | HOOK_MULT_BY_TIER.legacy — T1: 5.7×, T2: 4.0×, T3: 3.0× |
CollegeVine explicitly excludes legacy |
| Recruited athlete | HOOK_MULT_BY_TIER.athlete — T1: 4.5×, T2: 4.0×, T3: 3.5× |
CollegeVine excludes recruited athletes |
| Donor status | HOOK_MULT_BY_TIER.donor — T1: 12.0×, T2: 10.0×, T3: 8.0× |
CollegeVine excludes donor hooks |
| Feeder school effect | student.feeder_bonus (1.0–1.25×) additive in logit |
CollegeVine has no school-context model |
| Essay quality | essay_base (1–10) → per-application noise → essayScore |
CollegeVine acknowledges it "cannot evaluate essays" |
| First-gen status | 1.4× flat multiplier |
CollegeVine may model implicitly but doesn't confirm |
| Pell-eligible | 1.25× multiplier |
Not modeled by CollegeVine |
| Income bracket | Chetty 2023 residual: Q5 +15%, Q1 -8% (unhooked only) | CollegeVine doesn't model income effects on admission |
| Demonstrated interest | DI_BOOST per college × demonstratedInterestLevel × round multiplier |
CollegeVine excludes DI |
| Yield protection | yieldProtectionStrength penalty for overqualified unhooked applicants |
CollegeVine does not model yield protection |
| Consulting client boost | essay_base +0.5–1.0, ec_quality +0.3–0.6 |
CollegeVine wouldn't model its own competitive effect |
| International seat reserve | intlSharePct × intlShareShock reduces domestic capacity |
CollegeVine is domestic-focused |
| Phantom applicants | MODEL_SCALE = 0.013 — non-modeled applicants consume seats |
Not applicable to individual prediction |
| Seat constraints | Hard class-size limits, ED fill rates, round-by-round allocation | CollegeVine predicts probability, doesn't simulate market clearing |
| Archetype × school interactions | FIT_SCORES[archetype][college] (e.g., stem_spike +5 at MIT) |
CollegeVine groups majors but doesn't model archetype fit |
| Holistic noise | (rand() + rand() - 1) * 1.2 ≈ Normal(0, 0.7) in logit space |
CollegeVine may model uncertainty but doesn't disclose |
3. Academic Scoring
CollegeVine
Compares student's unweighted GPA and SAT/ACT against the 25th percentile of admitted students at each college. Course rigor (AP/Honors count) is factored in separately. The exact mathematical relationship is undisclosed.
As of ~2024, the engine distinguishes 3.80 vs. 3.85 (previously rounded). Uses IPEDS acceptance rates (confirmed 2020–2021 vintage with forecasting).
Key Difference
CollegeVine's academic component is a comparison to a single cutoff (P25). Ours is a continuous distance metric — a student 50 AI points above a college's median gets credit for that distance, while one 50 points below is penalized. This better captures the reality that admissions is not binary above/below a threshold.
4. Extracurriculars
CollegeVine — 4 Tiers / 12 Sub-Tiers
| Tier | Description | Examples |
|---|---|---|
| Tier 1 (National/International) | Rare, exceptional | D1 recruit, Regeneron finalist, USAMO winner, published novelist |
| Tier 2 (State/Regional) | High achievement | Club president, All-State, regional competition winner |
| Tier 3 (School-level leadership) | Minor leadership | VP/treasurer, small-scale projects |
| Tier 4 (General participation) | Membership | Club member, casual sports, volunteering |
CollegeVine says ECs account for ~35% of the chancing prediction. Users self-classify their activities.
Key Difference
CollegeVine relies on user self-classification of activities into tiers — inherently subjective and gameable. Our model generates EC quality from archetype distributions (a STEM spike at a boarding school will have higher EC quality than an average academic at a public school). The EC quality is correlated with structural position but not deterministic.
The ecBonus thresholds (≥9.0 and ≥7.5) create a step-function effect similar to CollegeVine's tier boundaries — but embedded in a continuous score rather than discrete categories.
5. Essays & Recommendations
CollegeVine
Not modeled. CollegeVine explicitly states essays and recommendation letters are excluded from the chancing engine. They acknowledge this is a major limitation — essays "typically separate those who get in from those who don't" at selective schools.
Key Difference
This is the single largest divergence between the two models. CollegeVine has a 0-point essay contribution; ours contributes up to 10 of ~80 raw points (12.5%). At selective schools where academic profiles are compressed (everyone has high AI), essay quality becomes a significant differentiator.
The consulting client effect (essay_base +0.5–1.0) is a unique feature — it models the real-world impact of spending $30K+ on Crimson Education or similar services.
6. Hooks & Demographic Factors
CollegeVine
| Hook | Modeled? | Details |
|---|---|---|
| Legacy | No | Explicitly excluded |
| Recruited athlete | No | Explicitly excluded |
| Donor/development | No | Explicitly excluded |
| First-generation | Unknown | Not confirmed |
| Race/ethnicity | Reduced | "Greatly reduced" post-SFFA; previously a factor |
| Gender | Yes | Input factor |
| Income | No | Not modeled |
| Geographic diversity | No | Only IS/OOS for publics |
Key Difference
CollegeVine's deliberate exclusion of legacy, athlete, and donor hooks means it systematically overpredicts chances for unhooked applicants and underpredicts for hooked applicants at selective schools. An unhooked student with a 3.9 GPA and 1550 SAT sees the same CollegeVine probability as a legacy athlete with the same stats — but in reality, the legacy athlete's admission probability at Harvard is ~20× higher (5.7× legacy × 4.5× athlete in logit space).
This is arguably the biggest flaw in CollegeVine's model for selective schools.
7. Early Decision / Early Action
CollegeVine
Added May 2022. Found: - ED multiplier: ~1.6× average at very selective schools - EA boost: 4–6% - REA boost: 6–8%
School-specific but methodology undisclosed.
Key Difference
CollegeVine gives a single-number ED boost. Our model simulates the full round dynamics: ED fills seats first, leaving fewer for RD. A student's RD probability depends not just on their profile but on how many seats were consumed in earlier rounds. This is a market-clearing effect that individual prediction tools fundamentally cannot capture.
Our ED multipliers also vary more widely — UChicago ~4×, Dartmouth 3.5×, Columbia 3.4× (from real data) vs. CollegeVine's ~1.6× average.
8. Demonstrated Interest
CollegeVine
Not modeled. Explicitly excluded.
Key Difference
At DI-heavy schools (Tulane, Northeastern, Wake Forest), this is a dominant factor. CollegeVine's exclusion of DI means it likely overpredicts RD chances at these schools for students who haven't visited/engaged, and underpredicts for students who have.
9. Application Strategy (List Building)
CollegeVine
Provides a School List Builder that recommends balanced reach/match/safety lists based on chancing probabilities. Uses the ~75-factor chancing score to classify schools.
Key Difference
CollegeVine recommends schools to real users. Our model simulates how students actually behave in building lists — including income-driven application count differences, archetype-specific targeting, and legacy school guaranteed inclusion. The stratified allocation (25/35/25/15) mirrors real counselor advice.
10. Calibration & Accuracy
CollegeVine — Published Calibration
| Predicted | Actual |
|---|---|
| 5% | 7.2% |
| 15% | 12.3% |
| 30% | 28.7% |
| 50% | 48.1% |
| 80% | 82.2% |
| 95% | 94.0% |
Aggregate calibration is good (within ~3pp at most bins). But: - Systematically overpredicts at selective schools (<20% acceptance rate) - Uses stale IPEDS data (2020–2021 baseline with forecasting) - Users report 65–71% at UC Berkeley, 82% at NYU for competitive CS, 66–78% at UCLA (15% real rate)
Key Difference
CollegeVine calibrates individual prediction accuracy ("when we say 50%, 48% get in"). We calibrate system-level outcomes ("does Harvard's simulated acceptance rate match its real rate? Does its simulated yield match? Does its ED fill rate match?").
Neither approach validates the other's claims. CollegeVine could be perfectly calibrated in aggregate while being wrong about every individual (if errors cancel). Our simulation could match every college's aggregate stats while being wrong about individual students (since it uses stochastic Bernoulli trials).
12. What CollegeVine Models That We Don't
1,500+ Colleges
We model 55. CollegeVine covers 30× more institutions, including the mid-selectivity schools (50–80% acceptance rates) where their calibration is actually strongest.
Real Individual Variation
CollegeVine profiles have actual personal data — the real student's GPA, their actual AP list, their specific extracurriculars. Our agents are synthetic, drawn from archetype distributions. A real student with a 3.85 GPA and 8 APs in STEM fields is more precisely represented in CollegeVine than as a "stem_spike from a boarding school with GPA 3.85."
Class Rank
CollegeVine uses class rank when available. We don't generate or score class rank — though it's implicitly captured by GPA relative to high school distribution.
Test-Blind vs. Test-Aware
CollegeVine added this distinction in Fall 2022 (relevant for UC system, Caltech). We don't differentiate — all colleges in our simulation use SAT scores.
14. Summary Scorecard
| Dimension | CollegeVine | Our Simulation | Winner |
|---|---|---|---|
| Academic scoring precision | GPA + SAT vs. P25 | Section-weighted AI delta | Ours — continuous distance, not threshold |
| Extracurricular modeling | 4-tier/12-sub-tier | Continuous 1–10 with tier bonuses | Tie — different tradeoffs (user input vs. archetype generation) |
| Essay modeling | None (0%) | 12.5% of raw score + per-app noise | Ours — CV's biggest acknowledged gap |
| Hook modeling (ALDC) | None | Tier-differentiated logit multipliers | Ours — legacy/athlete/donor are dominant at T1–T2 |
| Demographic modeling | Reduced post-SFFA | Full SFFA tier-differentiated + gender + income | Ours — more nuanced post-SFFA model |
| ED/EA modeling | ~1.6× average | Per-college overrides (1.2–8.0×) + round seat constraints | Ours — real data, market clearing |
| Demonstrated interest | None | Per-college DI boost × round multiplier | Ours — critical at Tulane/NEU/Wake |
| College coverage | 1,500+ | 55 | CollegeVine — 30× more schools |
| Individual precision | Real student data | Synthetic agents from archetypes | CollegeVine — actual profile vs. archetype |
| Market dynamics | None (isolated predictions) | Full market clearing, yield, waitlist, melt | Ours — the whole point |
| Transparency | Black box | Every coefficient inspectable | Ours |
| Aggregate calibration | Published ±3pp | Monte Carlo validated across 55 colleges | Tie — different calibration targets |
| Selective school accuracy | Systematically overpredicts | Calibrated with ELITE_BOOST + pool correction | Ours — by design |
| Accessibility | Free for any student | Research tool | CollegeVine — serves 2.4M users |
Sources
- CollegeVine chancing engine blog posts (see
collegevine.mdSources section) sim.js—computeAdmissionScore()(line 2383),initColleges()(line 2027),buildCollegeLists()(line 1514)- SFFA v. Harvard trial data (Arcidiacono expert testimony) — hook multiplier calibration
- Chetty 2023 (Opportunity Insights) — income residual
- Class of 2029 ED multiplier actuals —
research/ed_multipliers.json
Some sections containing simulation-specific implementation details have been omitted from this public version. The research data and analysis above is based on publicly available sources.