How should we build a theme for a new Drupal site?
Drupal core now has a Starter Kit that replaces Classy. This will copy some classy-esque styles and templates to set you up with some defaults, fixes, and BEM structure. The new theme is based on Stable, which is a copy of system libraries and templates from the X.0.0 of a major release. This ensures your site won't change if those underlying styles change, but you will now have to implement any fixes that were released. This tradeoff allows the starterkit theme to get ongoing development while generated themes are stable.
In order to build a theme from the ground up, there is a bit of work to do. Anything that Drupal can put on the page needs to have an appropriate style, and it's not always clear how to adjust it. Some core libraries are quite opinionated and developers need to know which ones to override or extend to do what they want. It can be hard to look at the code from a component perspective where you expect relevant pieces to be bundled together more clearly. Navigation and form elements can throw some surprises at you and since they are so coupled with the back end, themers can be confused about how they're styles will be displayed.
Building a theme from scratch is a more esoteric exercise than a practical one. Bootstrap based themes like Radix will give developers a major shortcut if they accept the decisions made for them by the community. Themes that support other front end frameworks are also a great way to implement them, and come with some documentation for how to get going. It takes some advanced knowledge to build a theme that loads code in a streamlined way with all the tools and tricks used today on the front end.
Buying a theme or starting with something like Olivero is a viable option if you want to minimize the work you have to do to make it look good. Watch out for the limitations that can bait you into something that looks easy but doesn't support your needs and leads to unexpected work.
Other themes built by agencies focus on component driven design and integrations with component libraries and style guides. These follow best practices for high quality, sustainable design, and solve a lot of the issues with how to handle the integration such as displaying drupal components in the design tool.
These themes that are built around external tools are great because they handle a lot of complexity for you. You will have to take some time to understand how it works and will be dependent on using the system, and limited by it's constraints. If you run into a snag that is a todo item for the supplier, you may have to resolve it yourself or wait for a solution. Early iterations of these could integrate basic features but took way too much time to get things working in both places, but many improvements have been made since them.
Supporting these themes are many modules like Components, Component Libraries, Progressively Decoupled Blocks, Single File Components, etc. These are different ways to define and integrate components between Drupal and a design system. This is an area that is in active development, and consolidation is still a hurdle to overcome before some kind of standard is set.
The goal is to have a way for non drupal developers to contribute to the front end, and for the team to be able to visualize the system as a whole. This will help with development and documentation. It doesn't matter if the solution is built in drupal or implements something like Storybook, as long as the requirements are handled and the developer experience is not annoying.