Scouting, Evaluations, and the Influence of the Environment
Talent identification and selection are the beginning, not the end.
NBA Draft season is in full swing, especially on social media, as last week concluded the draft lottery and the draft combine. Everyone on #DraftTwitter has posted his latest mock draft now that the order has been decided and the measurements have been collected.
One person chronicled his hits and misses over the last decade or more. An honest assessment of one’s evaluations and processes is a part of developing and improvement, however, as I replied, we can never know the accuracy of evaluations unless we are involved with the organization. Assessing and refining one’s processes, thinking, and biases improves one’s skills, but we never have all the information on the outside to judge our evaluations accurately.
During college and shortly thereafter, I argued with a local name scout from a popular network. He was convinced one high-school senior would be an NBA player, and therefore was a better recruit than another player who I believed would be a better college player. His evaluation centered on the potential of a 6’7 high-school ball-handler to remain a point guard in college and beyond and improve his shooting, whereas I bet on the 6’2 prototypical college point guard with a better shot who had a lower ceiling in terms of the NBA.
The shorter player was all-conference three seasons, led the conference in several statistical categories for at least one season, and had a better overall college career. The taller player was not drafted, but found his way onto an NBA roster for part of one season before spending his career in the G-League and overseas. He never averaged double figures in college.
The scout refused to concede that his evaluation was incorrect. He blamed the player for not working hard enough and the coach for not using him properly. He believed he would have performed better in a different environment. I saw these responses as a repudiation of private scouts, as there is little value in one’s evaluations if they can never be wrong or falsified, if there is always an excuse. One characteristic of pseudoscience is the lack of falsifiability, or the ability for a statement to be proven false. There was no consequence for being wrong, he had no skin in the game and his stature only grew, and no criticism was accepted because there was always an explanation. Anyone can be a scout if you are right even when you are wrong!
Scouting is not the end, unless the scout’s goal is simply to predict the draft order. Scouting, whether during high-school tryouts, college recruiting, or NBA drafting, is the beginning of a multifactorial process. The players chosen for the high-school team, recruited to an NCAA Division 1, or drafted into the NBA are not finished products. Their relative success or failure over the their high school, college, or NBA careers is not set at the time of selection, but determined by many factors after the evaluation, identification, and selection ends. We cannot account for all of these factors from outside the team environment, and even within the organizational environment, some things (injuries) are beyond one’s control, unpredictable, or unknowable during the evaluation and selection process.
Last spring, I visited my current club and evaluated the players. One player was on a tryout from another club, and I liked him. He joined our team this season, and I envisioned him as the starting point guard by season’s end. He previously played for a weaker club and was accustomed to iso-ing from the top and more or less doing whatever he wanted. He joined a much better team with much better teammates, and his goal was to be a point guard. He was not ready at the beginning of the season. He needed to improve, but as I tell players, that is why he was here.
He did not progress as quickly as hoped, but he was interested and asked questions. He was slowly developing into the player I envisioned. He had a lot of potential, but had flaws. He was a good defender, but often undisciplined. He had a decent shot, but his footwork and balance were poor. He was a capable driver, but his handle was loose. He was a good player against bad teams, but his goal was to be a point guard on the national team. He had to improve significantly to change from a scoring combo guard to a point guard, and from a bad local club team to a national-team caliber player.
I evaluated and identified him. He played in my system, in the environment I created. He was, in my eyes, making progress. His shot was improving and he hit two big three-pointers in our best win of the season and contributed a huge blocked shot. Afterward, I assured him I saw his positives and believed in him, even though he had not played as much as he had hoped to that point in the season (midpoint). I told him I realized I needed to let him be him a little bit too. I encouraged him and told him I still believed he would be our starting point guard in May at the finals. In my eyes, my evaluation and my ideas for him remained accurate, although progressing slower than anticipated. Everyone runs his own race.
The next day, his dad wrote a scathing letter to the club director in which, among other things, he said I ruined his son. He wanted to leave for another team. My club wanted to hold him hostage and put him on our second team. I told them to let him go. I am not the right coach for every player. I do not believe in sabotaging a 16-year-old simply because I am not the right coach or because his dad interferes too much. If his dad was a problem, as the director said, let him go and be a problem for another club. Do not punish the teenager.
Last I heard, he quit basketball and moved out of his dad’s house and into his mom’s. Does that mean my evaluation was incorrect? My coaching? Our style of play? Team environment? At the end of the day, we failed him or he failed on his own or his dad caused him to fail, but the result is the same. From the outside, the evaluation was incorrect; he was not the player I envisioned, and I was overly optimistic to believe he would be a starting point guard on a top 8 team. From the inside, the situation was far more complex than an inaccurate evaluation. Maybe I was right. Maybe his dad was correct. Maybe the truth (his actual ability and potential) is somewhere in the middle. We will never know.
As the scout suggested when I was in college, those outside an organization do not know a player’s work ethic, determination, resilience, and more. We do not know the coaching or the feedback the player receives. We do not know the vision the team has for the player and how the vision is articulated and accepted by the player. We do not know the aptitude of the player development coaches. Some players receive more opportunities than others because of the team’s personnel or the coach’s preferences or the team’s likelihood for success. Some players are affected more by their peers, positively or negatively. There are too many unknowns to fairly assess arm-chair evaluations.
Now, team personnel know most of these unknowns. They spend time with the player and interview previous coaches, trainers, families, and more. They know their coaching staff, the organization’s goals, the team’s system of play, and more. Therefore, teams should have a higher success rate evaluating, identifying, and selecting players for their organizations than #DraftTwitter.
Of course, these informed evaluations are never complete either, as the team may trade for a player who takes playing time from a recent draftee or the coach may be fired and replaced by a coach with a completely different philosophy. The general manager may be acquiring future talent and assets, not acquiring players who fit or will excel with the current roster. Some teams use the G-League much better than others. A father might be overbearing and ruin things for his son. There are many unknowns between selection and the final assessment of the evaluations.
Identifying and selecting talent is not a siloed process. Evaluations cannot occur in a vacuum. Talent identification must complement the talent development processes within the organization, which must complement the overall coaching and the team’s systems and styles of play. I evaluate players with my biases of how I would coach and develop the player, and how the player would fit into my desired style of play; this is different than evaluating players generally, and theoretically, I should be more accurate in my assessments for myself than general player evaluations with no context. The processes are interrelated, not separate entities; one influences and is influenced by the others. The entire environment eventually determines whether the evaluations and selections are proven accurate or incorrect.
As I wrote previously about Steph Curry and the Golden State Warrior environment:
Would Curry be Curry without Kerr? On any team, with any coach, he would be a great shooter and scorer. But, in a different environment with a different coach, would he be more like JJ Redick than Curry? How much of his brilliance is due to the environment?
The great players who seem like exceptions to hard and fast rules (Curry, Draymond Green, Dirk Nowitzki, Magic Johnson) find environments that allow them to break through the barriers of traditionalism.
With different coaches, Magic Johnson may have been Ervin Johnson, above-average power forward playing with his back to the basket on the block. With different coaches, Nowitzki may have been viewed as not strong enough to play as a back to the basket center and sent back to the European leagues. With different coaches, Green might be a TE in the NFL. With different coaches, Curry may have been more like Ray Allen than the best player in the game.
In each situation, the players would have been good to great players because of their skill, talent, size, attitudes, personality, and more. However, it took the players with those gifts to find the right environments to display those gifts. Rather than trying to change their personalities, or limit their skills to a more accepted range, they found coaches and environments that allowed them to utilize their skills to their utmost to see just how great and game-changing they could be.
Did the NBA miss on Curry because he was not the #1 pick in the 2009 NBA Draft? Was he the best player at the time? Did he develop at a faster and steeper rate than others after he was drafted? Should everyone have seen the potential for this steep ascension?
I have no idea. I was in New York for the 2009 NBA Draft, working on a documentary focused on Ricky Rubio and Brandon Jennings with Canadian filmmakers. I interview Sonny Vaccaro on the morning of the draft (or morning before) in his hotel suite. He was angry that it seemed Rubio would be selected over Jennings. I do not remember much about Curry; I was focused entirely on Rubio and Jennings.
What if Curry was selected #2 by Memphis instead of Hasheem Thabeet or #4 by Sacramento instead of Tyreke Evans or by Minnesota instead of Rubio or Johnny Flynn? Would he have had the same career? Would his shooting have redefined the NBA game over the last decade or more? Again, it is unknowable; we do not have a parallel timeline to test out these theories.
However, it is likely that Curry found in Kerr the perfect coach to unleash his shooting skill. The environment from Kerr’s system and coaching to teammates such as Klay Thompson, Draymond Green, Andrew Bogut, and Andre Igoudala to the strength coaches who rehabbed his ankles and developed his entire lower body coalesced perfectly to maximize his talents. The Grizz, Kings, and Twolves of that time were unlikely to provide the same environment.
Player evaluations and selections are snapshots at a specific moment in time. They do not freeze the player at that moment. Players change and develop, shaped by environmental (coaching, playing time) and internal (confidence, resilience) factors. Evaluations and selections must complement the overall environment (coaching, skill development, systems, style of play) to generate the greatest returns.