["R&D Management, EarlyView. ", "\nABSTRACT\nThese days, a wave of artificial intelligence applications is arriving in the way we manage innovation. But where are we now, and how can one use AI practically in innovation management? Against this background, we first extend an earlier literature review in text classification to demonstrate that the field remains scarce. We then address the current research gap by successfully conducting a machine learning project to evaluate ideas using text classification with transformer models. For this, we semi‐automatically extracted customer needs in the field of smart speakers by downloading a Twitter dataset containing 473,301 tweets. We classified 107,738 tweets using a machine learning model, identified 12,641 potential need‐containing tweets, and manually coded the first 1000. Our results can assist innovation managers and researchers in implementing high‐performing machine learning models to support their innovation management tasks in different ways. We documented the design choices for this machine learning project in detail, deriving guidelines for creating text classification projects tailored to the domain of innovation management.\n"]