Building a Portfolio You Can Talk To
I wanted my portfolio to be different. Not just another static site with sections to scroll through, but something that feels alive and conversational. So I built heyavi.me - a portfolio that literally talks back to you.
Instead of traditional navigation, visitors can click the mic button and have a natural conversation. Ask about my projects, experience, or even paste a job link - and the AI responds with both voice and dynamically generated UI components.
What It Does
The portfolio works as a traditional site if you prefer scrolling, but the magic happens when you interact with the AI assistant:
- Voice Conversations: Click the mic and speak naturally - "What projects have you built?" or "Tell me about your work at tambo"
- Dynamic UI Generation: The AI doesn't just talk - it shows relevant components like photo grids, project cards, or my resume
- Job Analysis: Paste any job posting URL and get an instant analysis showing how I match the requirements
- Smart Context: The AI adapts its responses based on who you are - student, recruiter, or fellow developer
The Technical Implementation
Core Stack
- Framework: Next.js 15 with App Router for optimal performance
- AI Integration: Tambo AI for the conversational layer
- Voice: OpenAI Whisper for speech-to-text, ElevenLabs for text-to-speech
- Database: PostgreSQL for blog comments, guestbook, and analytics
- Auth: NextAuth.js with GitHub OAuth
- Styling: Tailwind CSS with custom glassmorphic effects
- Crawling: Firecrawl for efficiently crawling links pasted into the chat
Voice Processing Pipeline
The voice interaction flow is surprisingly complex:
- Voice Activity Detection: Real-time audio analysis to detect when you start/stop speaking
- Smart Silence Detection: Waits for natural pauses before processing (1 second threshold)
- Audio Conversion: WebM to WAV format for Whisper API compatibility
- Submission Detection: Uses GPT-4 to determine if the transcript is complete
- Global Audio Management: Centralized system to handle TTS playback and prevent overlaps
Dynamic Component System
The AI can render 10+ different components based on context:
- PhotoGridSection - Shows my photos with captions
- ProjectsSection - Displays projects with descriptions
- ResumeOverview - Professional background + resume button
- JobAnalysisCard - Visualizes job matching results
- GuestbookDisplay - Shows visitor messages
- SocialMediaGrid - Contact links in a grid layout
The "Hire Me" Intelligence
When someone shares a job posting URL:
- Firecrawl extracts the job content
- AI analyzes requirements vs. my experience
- Generates a match score and talking points
- Shows a "JobAnalysisCard" with visual breakdown
- Provides a conversational, tailored pitch
Design Decisions
Minimal UI, Maximum Impact
- Floating Mic Button: Always accessible, with visual feedback for recording/speaking states
- Glassmorphic Panels: Semi-transparent overlays that don't obstruct content
- Progressive Disclosure: Start with just a mic, expand to show conversation history
- Keyboard Shortcuts: ⌘+K for power users
Voice-First, But Not Voice-Only
The site remains fully functional without voice:
- Traditional navigation in the header
- All content accessible via scrolling
- Text input option in the chat interface
- Fallbacks for unsupported browsers
Challenges & Solutions
Audio Feedback Loops: Solved with a global audio manager that tracks all playing audio and stops previous TTS when new messages arrive.
Mobile Voice Input: Browser limitations required careful handling of permissions and fallback options.
Component Generation Timing: The AI needed careful prompting to generate components at the right moments, not too eagerly.
What's Next
I'm continuing to refine the experience:
- More sophisticated job matching algorithms
- Multi-language support for global opportunities
- Voice personality customization
- Integration with more portfolio platforms
Try It Yourself
Visit heyavi.me and click the mic button. Ask me anything - about my projects, experience, or paste a job link to see the analysis in action.
The entire project is open source on GitHub. Feel free to fork it and build your own talking portfolio!
Key Takeaways
Building this taught me that AI can enhance personal websites without making them feel robotic. The key is maintaining authenticity while adding genuinely useful interactions. A portfolio should tell your story - mine just happens to tell it out loud.