Jobs Private USA → Technology → Software engineering → Principal Software Engineer - LLM Optimization

Principal Software Engineer - LLM Optimization

JPMorgan Chase · Jersey City, NJ · posted 40 days ago

In short: Principal Software Engineer - LLM Optimization at JPMorgan Chase in Jersey City, NJ, posted 40 days ago.

By AtiveSite · Published 07/11/2026 · Updated 08/11/2026 · Every figure is linked to its source below.

JPMorgan Chaseemployer

Jersey City, NJlocation

Software engineeringsub-niche

40 days agoposted

🆕 Work guides: live jobs + official data, law, permits, pay and cost of living. United States · New Jersey · Hudson County · Jersey City · all 100 cities + map → · all 21 counties → · New Jersey rankings · all 51 states → · 200 largest companies · 240 best job niches · Federal jobs in New Jersey

The job

LIVE Live feed (private-sector employer) Source: The Muse public API · Data: listed 2026-09-29

Employer	JPMorgan Chase
Location	Jersey City, New Jersey
Field	Technology › Software engineering
Level	Senior Level
Source category	Software Engineering
Posted	2026-09-29 (40 days ago)
Work arrangement	On-site at the location listed
Pay	Not published in the source listing

Job description

At JPMorganChase, we are building the infrastructure that powers the next generation of enterprise AI - and we need the best minds in LLM inference to help us do it. This is your opportunity to work at the intersection of cutting-edge machine learning and large-scale production systems, directly influencing how one of the world's largest financial institutions deploys and optimizes AI at scale.

As a Principal Software Engineer at JPMorganChase within the AI/ML Data Platform team, you will serve as the firm's deepest technical voice on LLM inference performance - owning optimization strategy, benchmarking rigor, and efficiency at scale. You will work directly with senior engineering leadership to shape how our platform evolves, ensuring every model we serve is fast, cost-efficient, and production-ready. This is a high-visibility individual contributor role where your technical decisions will have direct, measurable impact on the firm's AI capabilities

Job Responsibilities

Own systematic benchmarking and performance characterization across all production LLM workloads. Establish reproducible baselines, catch regressions early, and quantify the impact of every configuration change before it touches production
Design and execute quantization experiments - FP8, INT8/INT4 (GPTQ/AWQ), next-generation precision formats on current hardware - measuring accuracy delta, throughput improvement, memory reduction, and cost-per-token impact
Drive speculative decoding strategy across the model portfolio: draft model, n-gram, and multi-token prediction approaches. Own acceptance rate measurement and per-workload configuration recommendations
Build and maintain a GPU efficiency scorecard: utilization, memory headroom, cost per 1K tokens, and waste identified - giving leadership a data-driven view of platform efficiency at all times
Benchmark our platform against external providers and published industry numbers - know what good looks like, and close the gap
Lead inference engine upgrade evaluations: new scheduler architectures, async tensor parallelism, disaggregated prefill/decode, advanced speculative decoding - systematic validation before production promotion
Collaborate with the EKS and disaggregated serving teams on KV-cache optimization, prefix caching strategies, and multi-node serving architecture
Design and run GPU chaos engineering: induced failure scenarios, hardware diagnostic monitoring, detection and recovery measurement
Architect and govern agentic AI-enabled engineering workflows (using enterprise-authorized tools within the work environment) to improve delivery speed, code quality, and operational outcomes at scale (e.g., AI-driven PR review assistance, test generation/maintenance, release readiness checks, incident triage and root-cause acceleration), while defining guardrails for validation, security, resiliency, and reuse across teams.
Apply knowledge of tools within the Software Development Life Cycle toolchain, including enterprise-authorized AI-assisted development and automation capabilities, to improve the value realized by automation at scale.

Required qualifications, capabilities, and skills

Formal training or certification on software engineering concepts and 7+ years applied experience
Deep, hands-on experience with LLM inference systems - vLLM, TensorRT-LLM, SGLang, LLM-D or equivalent production serving engines
Strong grasp of GPU memory architecture: KV cache sizing and dynamics, memory-bandwidth vs compute bottlenecks, the practical implications of quantization at inference time
Experience with quantization techniques and their real-world tradeoffs at scale
Familiarity with speculative decoding and the variables that drive acceptance rates in production workloads
Rigorous benchmarking instincts - GuideLLM, custom harnesses, or equivalent. Every claim has a number behind it
Comfort operating in cloud GPU infrastructure at scale (AWS; EKS, managed inference services)
Demonstrated awareness of the LLM inference competitive landscape, with a track record of applying industry benchmarks to drive platform improvements communicate technical trade-offs clearly to senior engineering and business stakeholders - this role presents upward regularly
Demonstrated experience designing and leading adoption of agentic AI-enabled development practices (using enterprise-authorized tools within the work environment) across teams, including setting standards for human-in-the-loop validation, auditability/traceability of changes, and secure handling of sensitive data.
Strong understanding of responsible AI use and control expectations in engineering workflows, including security/resiliency implications, data sensitivity, and risk-based governance; ability to influence senior technical leaders on safe scaling patterns and reuse.

Preferred qualifications, capabilities, and skills

Experience with disaggregated prefill/decode serving architectures, GPU hardware diagnostics (DCGM/NVML/XID event tracking), ML observability and production monitoring

FEDERAL DEPOSIT INSURANCE ACT: This position is subject to Section 19 of the Federal Deposit Insurance Act. As such, an employment offer for this position is contingent on JPMorganChase's review of criminal conviction history, including pretrial diversions or program entries.

ABOUT US

JPMorganChase, one of the oldest financial institutions, offers innovative financial solutions to millions of consumers, small businesses and many of the world's most prominent corporate, institutional and government clients under the J.P. Morgan and Chase brands. Our history spans over 200 years and today we are a leader in investment banking, consumer and small business banking, commercial banking, financial transaction processing and asset management.

We offer a competitive total rewards package including base salary determined based on the role, experience, skill set and location. Those in eligible roles may receive commission-based pay and/or discretionary incentive compensation, paid in the form of cash and/or forfeitable equity, awarded in recognition of individual achievements and contributions. We also offer a range of benefits and programs to meet employee needs, based on eligibility. These benefits include comprehensive health care coverage, on-site health and wellness centers, a retirement savings plan, backup childcare, tuition reimbursement, mental health support, financial coaching and more. Additional details about total compensation and benefits will be provided during the hiring process.

We recognize that our people are our strength and the diverse talents they bring to our global workforce are directly linked to our success. We are an equal opportunity employer and place a high value on diversity and inclusion at our company. We do not discriminate on the basis of any protected attribute, including race, religion, color, national origin, gender, sexual orientation, gender identity, gender expression, age, marital or veteran status, pregnancy or disability, or any other basis protected under applicable law. We also make reasonable accommodations for applicants' and employees' religious practices and beliefs, as well as mental health or physical disability needs. Visit our FAQs for more information about requesting an accommodation.

JPMorgan Chase & Co. is an Equal Opportunity Employer, including Disability/Veterans

ABOUT THE TEAM

Our professionals in our Corporate Functions cover a diverse range of areas from finance and risk to human resources and marketing. Our corporate teams are an essential part of our company, ensuring that we're setting our businesses, clients, customers and employees up for success.

Explore around this job

Field: Technology Sub-niche: Software engineering All jobs at JPMorgan Chase Jobs in New Jersey Jobs in Jersey City, NJ Jersey City city guide New Jersey state guide Hudson County economy

View this job and apply on The Muse →

You apply on The Muse or on the employer's page; AtiveSite does not receive or process applications and is not affiliated with The Muse. The description above is shown as the source published it.

About the role

This is a Senior Level role in the Technology field (Software engineering) at JPMorgan Chase. Senior roles usually expect substantial experience, ownership of results and often the ability to guide others.

We could not match this job title to one occupation in the official wage statistics, so no pay estimate is shown. Pay is not in the source listing; ask the employer for the range and compare in the 240 job niches.

Does the pay work here? (estimate)

Gross salary a year Filing State income tax rate %

Estimate for a wage earner: 2026 federal brackets and standard deduction (IRS), Social Security 6.2% up to $184,500, Medicare 1.45% (+0.9% above $200,000 single / $250,000 married), and a state rate you set; set your own state tax rate. It ignores credits, pre-tax deductions, local taxes and benefits. Sources: IRS: tax year 2026 inflation adjustments · SSA wage base per Federal Register notice 2025-19763.

How to prepare and behave

This is editorial guidance, not information from the employer.

Read the whole description and match your examples to its requirements; use the same words the posting uses where they are true for you.
Research the employer before the interview: what it does, who its customers are and recent news; prepare two or three questions about the work.
Arrive early (or log in early for a video call), dress one step above the daily dress code, and test your camera and audio beforehand.
Answer with short stories: the situation, what you did and the result. Give numbers when you have them and never invent them.
Be honest about gaps; say what you did to learn and how fast you can close them.
Ask about the team, the first 90 days, how success is measured and the schedule; do not open with salary and time off.
When pay comes up, know the market range for the occupation and your lowest acceptable offer before you answer; ask for the full package in writing.
Send a short thank-you message within a day, mentioning one thing you discussed.
Have code, data work or projects you can show, and be ready to think aloud: many technical interviews test how you reason, not only the answer.

Know your rights as a candidate and an employee:

The city

The job is in Jersey City, NJ. Below is what the official statistics say about living and working there.

How Jersey City compares. Among the 100 largest places in New Jersey, Jersey City is #2 by population, #93 for long commutes (#1 = fewest), #91 for homeownership (#1 = highest), #19 for remote work (#1 = most). Its county, Hudson County, is #3 of 21 counties for average pay ($94,901).

The place in numbers

Jersey City has 294,078 residents (ACS 2020–2024), about 3.1% of the population of New Jersey. It covers 15 square miles of land, which works out to roughly 19,940 people per square mile. The median age is 34.7, 13% younger than the state median (34.7 against 40.1).

Housing

The median gross rent is $2,007 a month (rent plus utilities), 17% above the state median ($2,007 against $1,720). Over a year that is 24.6% of the median household income. US housing guidance treats housing as affordable when it takes 30% of income or less, so the typical household is under that line. The median home value is $566,900, which is 5.8 times the median household income ($97,710); a ratio above 5 usually means buying is out of reach without savings or a second income. 27.9% of households own their home, 56% below the state average (27.9% against 63.8%). A low ownership share points to a rental-heavy market with more apartments and shorter stays.

Work and pay

The New York-Newark-Jersey City, NY-NJ metropolitan area has 9,492,000 jobs in the BLS wage survey, and the median pay across all occupations is $61,430 a year, 20% above the national median ($61,430 against $50,980). The biggest occupation groups are office and administrative support (1,109,360 jobs, median $55,080); healthcare support (843,730 jobs, median $40,100); sales and related (751,110 jobs, median $47,540). The area is most specialised in healthcare support (1.8 times) and legal (1.7 times), where the share of workers is well above the national share. The best-paying groups with at least 5,000 jobs are management ($171,990) and legal ($150,990). In 2026M09 the unemployment rate of Hudson County was 3.8%. 70.5% of residents aged 16 and over are in the labour force. The city lies in Hudson County; its jobs, pay by industry and GDP are on the county page.

Getting around

Among workers, 26.1% drive alone, 36.1% use public transit, 5.5% walk, 24.7% work from home. 36.0% of workers commute 45 minutes or more, 54% above the state average (36.0% against 23.4%). Remote work is more common than in the state as a whole, which widens the set of employers a resident can work for without moving.

People and education

54.8% of adults aged 25 and over hold a bachelor's degree or higher, 26% above the state share (54.8% against 43.6%). 41.7% of residents were born outside the United States, 74% above the state share (41.7% against 23.9%). The poverty rate is 15.6%, 61% above the state rate (15.6% against 9.7%).

Climate

The warmest month is Jul, averaging 76°F (24.4°C), and the coolest is Jan at 30°F (-1.1°C). About 1,227 mm (48 inches) of precipitation falls in a year; Jun is the wettest month (117 mm) and Feb the driest (80 mm). These values come from NASA satellite-based climatology on a grid of about 50 km, so they describe the wider area, not a weather station in the city.

Nearby cities to compare

Closest cities in this guide: Hoboken (3 miles, rent 46% higher); Bayonne (4 miles, rent 16% lower); Kearny (4 miles, rent 14% lower); Union City (4 miles, rent 23% lower). Rent differences are against Jersey City; use the Compare tab to put any two cities side by side.

Full guide to Jersey City →New Jersey guide All jobs in Jersey City

Official links for Jersey City

These links go to official sources and open in a new window. The first one is the government's or company's own site as listed in Wikidata (property "official website"); check it before relying on it.

Where to live

CITY Official statistics Source: U.S. Census Bureau, American Community Survey 2020–2024 5-year (via Census Reporter) · Data: ACS 2020–2024

Places within 50 miles (straight line) of Jersey City, NJ that have a guide, with what living there costs. No pay estimate is shown because the title did not match one occupation.

Sort:

Place ⇅	Distance from the job city ⇅	Median rent ⇅	Median home value ⇅	Median household income ⇅	Commute 45+ min ⇅	Rent as % of occupation pay ⇅	Open jobs there ⇅
Jersey City, NJ	0 mi	$2,007	$566,900	$97,710	36.0%	n/a	Jobs
Hoboken, NJ	3 mi	$2,938	$895,100	$180,579	38.7%	n/a	—
Bayonne, NJ	4 mi	$1,682	$460,100	$83,887	31.0%	n/a	—
Kearny, NJ	4 mi	$1,720	$459,400	$79,154	26.6%	n/a	—
Union City, NJ	4 mi	$1,537	$471,600	$64,310	27.9%	n/a	—
Secaucus, NJ	5 mi	$2,391	$571,900	$139,821	33.9%	n/a	Jobs
Harrison, NJ	5 mi	$2,226	$505,800	$93,013	42.0%	n/a	—
Newark, NJ	6 mi	$1,392	$373,700	$52,060	25.1%	n/a	Jobs
West New York, NJ	6 mi	$1,716	$465,600	$71,729	30.6%	n/a	—
North Arlington, NJ	6 mi	$1,738	$474,500	$94,201	29.5%	n/a	—
New York, NY	7 mi	$1,821	$777,600	$80,483	41.2%	n/a	Jobs
Elizabeth, NJ	7 mi	$1,523	$449,300	$66,555	14.5%	n/a	—
Rutherford, NJ	8 mi	$1,966	$603,600	$143,153	33.0%	n/a	—
Fairview, NJ	8 mi	$1,694	$511,400	$63,606	25.4%	n/a	—
East Orange, NJ	9 mi	$1,441	$365,900	$60,830	29.9%	n/a	—
Cliffside Park, NJ	9 mi	$1,837	$588,200	$93,452	33.1%	n/a	—
Edgewater, NJ	9 mi	$2,579	$715,700	$124,050	43.6%	n/a	—
Palisades Park, NJ	10 mi	$2,009	$719,600	$94,743	26.9%	n/a	—
Passaic, NJ	11 mi	$1,465	$429,900	$55,371	16.6%	n/a	—
Linden, NJ	11 mi	$1,731	$419,700	$92,915	25.3%	n/a	—

Career paths

We could not match this title to one occupation, so here are the other sub-niches in Technology, where your skills may carry over:

Cybersecurity (366) Data, analytics and AI (1991) Cloud, IT and infrastructure (504) Product management (412)

Distances

CITY Calculation (straight line) Source: U.S. Census Bureau Gazetteer (coordinates) · Data: 2024

How far Jersey City, NJ is from the 20 largest US cities, in straight-line miles (not driving distance). Click a column to sort.

Sort:

From ⇅	Straight-line distance (miles) ⇅	Population of that city ⇅
New York, NY	7 mi	8,483,844
Philadelphia, PA	74 mi	1,579,706
Columbus, OH	472 mi	914,802
Charlotte, NC	529 mi	903,844
Indianapolis, IN	640 mi	885,860
Chicago, IL	711 mi	2,711,226
Jacksonville, FL	834 mi	977,670
Oklahoma City, OK	1,321 mi	697,125
Dallas, TX	1,365 mi	1,307,930
Fort Worth, TX	1,395 mi	963,194
Houston, TX	1,415 mi	2,328,253
Austin, TX	1,507 mi	979,539
San Antonio, TX	1,578 mi	1,479,835
Denver, CO	1,618 mi	718,877
Phoenix, AZ	2,134 mi	1,642,323
Seattle, WA	2,400 mi	754,195
San Diego, CA	2,420 mi	1,389,526
Los Angeles, CA	2,452 mi	3,857,263
San Jose, CA	2,543 mi	990,138
San Francisco, CA	2,595 mi	830,235

Nearby cities with open jobs (within 120 miles)

Sort:

City ⇅	Straight-line distance ⇅	Open jobs ⇅
Brooklyn, NY	7 mi	1
Manhattan, NY	7 mi	3
New York, NY	7 mi	2,049
Iselin, NJ	16 mi	2
Mahwah, NJ	26 mi	2
Deer Park, NY	39 mi	17
Darien, CT	40 mi	11
Lakewood, NJ	43 mi	2
Bethlehem, PA	69 mi	3
Fort Washington, PA	71 mi	2
Marlton, NJ	72 mi	13
Philadelphia, PA	74 mi	165
Atlantic City, NJ	94 mi	5
Wilmington, DE	103 mi	2
Newark, DE	114 mi	8

The employer

JPMorgan Chase is the employer in this listing. See all its open jobs here →

size: Large Size; industries: Financial Services; locations on The Muse: Manhattan, NY.

It is among the 200 largest US-headquartered employers in our list, with about 318,512 employees worldwide (2026). Company page with headquarters, state economy and official links →

Official links for JPMorgan Chase & Co.

The employer's profile on The Muse →

What it is like to work at a company (culture, schedules, benefits) is not in the source data, so we do not describe it. Read the description and ask the questions under "How to prepare".

Other open jobs at JPMorgan Chase

Vice President Product Partnerships - Commerce Payments

JPMorgan Chase · New York, NY · Senior Level · Treasury, payments and risk · posted yesterday

Treasury Management Officer - Technology & Disruptive Commerce - Vice President

JPMorgan Chase · New York, NY · Senior Level · Treasury, payments and risk · posted yesterday

Vice President, Cobrand CX Strategy & Insights

JPMorgan Chase · New York, NY · Senior Level · Marketing and advertising · posted yesterday

Sales And Marketing Analytics Lead-Vice President

JPMorgan Chase · Columbus, OH · Senior Level · Data, analytics and AI · posted yesterday

Payments Product Control - Associate

JPMorgan Chase · New York, NY · Senior Level · Treasury, payments and risk · posted yesterday

Experience Design Vice President - Agentic Treasury

JPMorgan Chase · New York, NY · Senior Level · Treasury, payments and risk · posted yesterday

Quantitative Trading & Research - EM Rates - Associate

JPMorgan Chase · New York, NY · Mid Level · Treasury, payments and risk · posted yesterday

J.P. Morgan Wealth Management - Vice President, Liquidity & Banking Specialist

JPMorgan Chase · Chicago, IL · Senior Level · Banking and wealth advisory · posted yesterday

Similar jobs

Same sub-niche: Software engineering

Principal Architect- Hypervisors, Operating Systems, & Datacenter

JPMorgan Chase · Jersey City, NJ · Senior Level · Software engineering · posted yesterday

Senior Lead Software Engineer

JPMorgan Chase · Jersey City, NJ · Senior Level · Software engineering · posted 4 days ago

IT Business Management Senior Analyst

DTCC · Jersey City, NJ · Senior Level · Software engineering · posted 8 days ago

Scrum Master Senior Associate

DTCC · Jersey City, NJ · Senior Level · Software engineering · posted 8 days ago

Director of Software Engineering, Acceleration & Modern Engineering Practices (Asset Management)

JPMorgan Chase · Jersey City, NJ · Senior Level · Software engineering · posted 8 days ago

Software Engineer - Java Full Stack

DTCC · Jersey City, NJ · Mid Level · Software engineering · posted 9 days ago

Lead Software Engineer, Cloud Platforms

JPMorgan Chase · Jersey City, NJ · Senior Level · Software engineering · posted 18 days ago

Software Development Engineer in Test (SDET) - Java.

DTCC · Jersey City, NJ · Senior Level · Software engineering · posted 22 days ago

More jobs in Jersey City, NJ

Employee-Facing Technology Products Communications Lead - Vice President

JPMorgan Chase · Jersey City, NJ · Senior Level · Communications and PR · posted yesterday

Sr Lead Security Engineer

JPMorgan Chase · Jersey City, NJ · Senior Level · Cybersecurity · posted yesterday

Principal Architect- Hypervisors, Operating Systems, & Datacenter

JPMorgan Chase · Jersey City, NJ · Senior Level · Software engineering · posted yesterday

Software Engineer III- AI/ML Platform

JPMorgan Chase · Jersey City, NJ · Senior Level · Data, analytics and AI · posted yesterday

Executive Product Solutions Director (Data, AI & GenAI Architecture)

JPMorgan Chase · Jersey City, NJ · Senior Level · Data, analytics and AI · posted yesterday

Product Manager - Employee Compliance - Vice President

JPMorgan Chase · Jersey City, NJ · Senior Level · Product management · posted 2 days ago

All Software engineering jobs → · All Technology sub-niches →

FAQ

What is the Principal Software Engineer - LLM Optimization job at JPMorgan Chase?

A senior level role in technology (software engineering) listed 40 days ago. The full description is on this page and on The Muse.

Where is this job?

Jersey City, NJ.

How do I apply?

Use the "View this job and apply on The Muse" button. AtiveSite does not receive applications.

How much does it pay?

The listing does not publish pay, and we could not match the title to one official occupation.

What is the cost of living there?

Median rent in Jersey City is $2,007 a month and median household income $97,710 (ACS 2020–2024).

How old is this listing?

It was posted on 2026-09-29 (40 days ago). Jobs older than 90 days are not shown.

Are there similar jobs?

Yes: browse software engineering jobs in New Jersey.

Sources and terms

Job data: The Muse public API (listing dated 2026-09-29). The job disappears from this site when it disappears from the source.
Wages and projections: BLS OEWS (May 2025) and Employment Projections 2025–35.
City and county figures: Census Bureau ACS 2020–2024, BEA, BLS QCEW and LAUS, NASA POWER climatology; distances from Census Gazetteer coordinates.

AtiveSite is not affiliated with or endorsed by The Muse, the employer or any government agency, and does not process applications. Verify every detail with the employer before acting.

Sources

Jobs in other countries

🇺🇸 United States 🇩🇪 Germany 🇸🇪 Sweden 🇸🇬 Singapore