Open Source 2025 Notes

Throwback Thursday: Old School Optimization Using Newfangled Machine Learning

Linear Programming

• "Programming" in the sense of scheduling.
• Inequalities carve out regions of space. These are called "constraints".
• Objective function: goal we are trying to maximize.

Linear Optimization Facts

• Solvable when bounded.
• Min-Max duality.
  • As an example - RPGs: Max strength == min weakness
• Deterministic - guarantees best answer given the input. Is susceptible to garbage input.

When to use

• Making the best choice out of many available while satisfying constraints.
  • Resource allocation.
  • Matching.
  • Routing problems.
  • Inventory decisions.
• Best fit for batch decisions under constraints.

Expected Outcomes

• Better use of resources.
• Stable operations planning.
  • Better inventory management

Pros of Linear Programming

• Measured risk, explainable, and respects your contraints. Can be built quickly with a small team.
• Plays nice with forecasted values: making best prediction with the data available.
• Any vertex of the feasible region has a cone of normal vectors where it is the optimal solution.
• Strength of forecast affects the feasible region.

How to Make One?

• Pyomo functions like an ORM for interfacing with solvers.
• CBC (COIN-OR Branch Cut) to solve.

React at Work Post-Create-React-App

• Provided testing, bundling, and a dev server built-in.
• "Kill it with Fire" book looks interesting.
• Create React App officially deprecated in February 2025.
  • React 19 doesn't work well with it.
  • React is slowly becoming its own framework instead of just acting as a component in a bring-your-own-framework model.

Alternatives

Next.js

Cons:

• Tight coupling of backend and frontend.
• Built-in server runtime.

Remix

• Also tightly coupled.
• Gnarly nested routing structure.

Bun

• Really fast bundler.
• Too much magic.
• Incomplete ecosystem compatibility.

Vite

• Fast, flexible, and friendly.
• Pretty easy to migrate from CRA.
• Static by default.
• Unbundled dev server, optimized builds.
• Can use existing rollup plugings.

Open Source Tooling and Best Practices to Improve Vulnerability Management

• VM: Vulnerability Management

• --Identification -> Reporting -> Evaluation -> Prioritization -> Remediation -- ∧ | | ∨

• Competency trap: people don't use new tools because building competency in a new tool is challenging.
• Mend Renovate: formerly Renovate.
• Renovate bot can work off of dependencies defined in comments in a dockerfile.

Beyond the Chatbot: Delivering Business Value with LLMs

• According to IBM "Institute of Business Value", only 25% of AI initiatives have delivered expected ROI.
  • 16% have scaled enterprise wide.
  • IBM's Advice: ignore FOMO, lean into ROI.
• When to chatbot:
  • Onboarding users (employees/customer) to complex systems.
  • When users are lost, confused, or not even sure what they need.
• Need metrics
  • conversion rate
  • % requests routed to a human
  • manual time saved
  • response/execution times.
• When not to chatbot:
  • Simple forms
  • When users are experts
  • LLM Assisted Automation (agents) can provide value by performing tasks on triggers.
  • Using AI to capture business that you don't have the number of people to take in the business?!?!
• Solving problems with GenAI:
  • Classifying unstructured data.
    • What kind of document is it? What needs to be done with it?
  • Convert unstructured so structured data?
    • Parsing quote requests, emails, etc.
  • Translate similar data between systems:
    • Referencing information from external vendor systems.
• Why was something not already automated?
  • Complex SOPs (Standard Operating Procedures).
  • Unstructured data
  • Parsing information from multiple internal/external services

Measuring GenAI Solutions

LLM Evals

"Answer true or false" is the key.

• Example:

SYSTEM: Act as a metallurgy expert. Do not explain your answer. Answer true or false.

USER: Steel is an alloy of iron and carbon?

Easy to apply to any business domain. Ask experts: what are the toughest questions you're asked? What are the perfect 20 year veteran answers? Then let the expert decide when the AI is trustworthy enough.

Metrics Driven Development

• Answer correctness
  • True/false, multiple choice questions
  • More complex questions:
    • Determine ground truth
    • Take intersection of model answers and ground truth and divide by intersection + model + ground truth.
      • AnswerIntersections / (AnswerIntersections + Ground Truth + Model Answers)
• Faithfulness - degree to which outputs align with facts
• Relevancy
• Context recall
• Arize Phoenix: LLM evaluation tool
• guardrailsai.com

These tools are generally used to test existing LLM's, not to fine tune or create a new model. Most things that are changed by the LLM implementer are the context that is fed in, or the prompt that is given.