AI Engineer

Join Telavox as an AI Engineer 🧪 Do you want to be the person who decides which models our AI products run on, and prove it with data? Telavox builds AI into the communication platform our customers already use: transcripts and summaries of their phone calls, AI agents that answer calls and messages, and the quality controls around them. We are looking for an AI Engineer who owns how good all of that is, across every AI product we ship. In this role, placed in Malmö or Stockholm, you'll measure the quality of what our models produce, build the evaluation pipelines that measure it continuously, and choose the models and prompts we run in production. You'll work in a small AI domain with two engineering teams, shaping priorities together with the product manager day to day, and your findings decide what goes live. If you'd like to take the next step in your career and are excited by the challenge of making AI measurably better, this is your opportunity! About the job Our AI products are live and growing, sold both directly and through telecom operators across Europe. Every one of them depends on models we do not train ourselves, so the choice of model, the prompts, and the quality metrics are decided by evidence from our own calls. This role owns that evidence. You'll work with the backend engineers who run the production services, with the team building our AI agent platform, and with customers and partners whose requirements shape what good means for them. Our stack: Python for evaluation and analysis, Java and Spring Boot in the production services, Google Vertex AI with Gemini alongside other model providers. Your day-to-day work may include: Building and running evaluation pipelines for transcription, summarisation and conversational agents: reference sets from real calls, automated metrics, LLM-as-judge, and validation that the metrics themselves can be trusted. Benchmarking candidate models, hosted and open, and making the call on what runs in production, with quality as the gate and cost as the constraint. Designing and evaluating the prompts and structured outputs behind our products, including the summary instructions customers can define themselves. Evaluating models that can be deployed regionally for customers with data-residency requirements. Putting quality measures into production, both ours and the ones customers define themselves, and presenting what they show to engineers, product, and leadership. About you You're a hands-on engineer who prefers evidence to opinion, and who is comfortable telling the team that the newer model is not better. You build the tooling you need rather than waiting for it; you can read a distribution and spot when a metric is measuring the wrong thing; and you explain your results so that people who did not run the experiment can act on them. You enjoy a small team where your work is visible in the product within weeks. Key skills and experience we're looking for: Hands-on work evaluating LLM or machine-learning model output in a real product: building datasets, defining metrics, running benchmarks, and turning them into decisions. Strong Python and data-science tooling, and experience analysing data to make product and business recommendations Experience with prompt design and structured outputs for LLMs in production The ability to communicate findings clearly, in writing and in presentations Proficient in English, both spoken and written Good-to-have skills: Speech and audio experience: transcription metrics such as WER and CER, diarization, forced alignment, and timestamp quality Java or another backend language, for integrating evaluations into production services Experience running open models yourself, such as deploying or evaluating self-hosted speech or language models on GPUs Familiarity with conversational AI or real-time voice agents Language proficiency in Swedish, Arabic, or other languages spoken in our main markets. Judging the quality of transcripts and agent conversations on customer calls is much easier in a language you speak Join us at Telavox 💚 Since Telavox's journey started in 2002, we have strengthened our position as the leading forward-thinking communication platform for businesses. As a digital mobile operator, we develop and own our all-in-one communication platform, which integrates telephony, PBX, messaging, meetings, and contact centers, giving businesses a smarter way to connect. Today, we’re a thriving company with 1.9 billion SEK in revenue and 500+ Telavoxers across nine countries. We embrace AI and automation to push the boundaries of business communication. We offer flexible work options and adaptable hours, giving you the freedom to balance life and career while staying connected to our vibrant Telavox culture. Read more about our new HQ in Malmö 🏠! How to apply! At Telavox, we don't focus on fitting in; we focus on making room for everyone. We’re always on the lookout for great talent, so applications are reviewed continuously. If you're ready to be part of an innovative, AI-driven telecom company, apply today with your CV and cover letter in English.  Learn more about what we do! ⬇️