research-article
Authors: Chunggi Lee, Tica Lin, Hanspeter Pfister, Chen Zhu-Tian
IEEE Transactions on Visualization and Computer Graphics, Volume 31, Issue 1
Pages 12 - 22
Published: 10 September 2024 Publication History
Metrics
Total Citations0Total Downloads0Last 12 Months0
Last 6 weeks0
New Citation Alert added!
This alert has been successfully added and will be sent to:
You will be notified whenever a record that you have chosen has been cited.
To manage your alert preferences, click on the button below.
Manage my Alerts
New Citation Alert!
Please log in to your account
- View Options
- References
- Media
- Tables
- Share
Abstract
As basketball's popularity surges, fans often find themselves confused and overwhelmed by the rapid game pace and complexity. Basketball tactics, involving a complex series of actions, require substantial knowledge to be fully understood. This complexity leads to a need for additional information and explanation, which can distract fans from the game. To tackle these challenges, we present Sportify, a Visual Question Answering system that integrates narratives and embedded visualization for demystifying basketball tactical questions, aiding fans in understanding various game aspects. We propose three novel action visualizations (i.e., Pass, Cut, and Screen) to demonstrate critical action sequences. To explain the reasoning and logic behind players' actions, we leverage a large-language model (LLM) to generate narratives. We adopt a storytelling approach for complex scenarios from both first and third-person perspectives, integrating action visualizations. We evaluated Sportify with basketball fans to investigate its impact on understanding of tactics, and how different personal perspectives of narratives impact the understanding of complex tactic with action visualizations. Our evaluation with basketball fans demonstrates Sportify's capability to deepen tactical insights and amplify the viewing experience. Furthermore, third-person narration assists people in getting in-depth game explanations while first-person narration enhances fans' game engagement.
References
[1]
23 amazing nba viewership statistics in 2024. “https://playtoday.co/blog/stats/nba-viewership-statistics/”. Accessed on March 25, 2024.
[2]
5 clever nba set plays and strategies explained. “https://www.youtube.com/watch?v=Fd3MzuHKHHI”. Accessed on March 21, 2024.
[3]
6 genius nba plays explained. “https://www.youtube.com/watch?v=lpR9Fp84XPw&;t=146s”. Accessed on March 21, 2024.
[4]
Court vision. “https://www.clipperscourtvision.com/”. Accessed on March 25, 2024.
[5]
The most popular sports in the world. “https://www.worldatlas.com/articles/what-are-the-most-popular-sports-in-the-world.html”. Accessed on March 25, 2024.
[6]
Nba sportvu dataset. “https://paperswithcode.com/dataset/nba-sportvu”. Accessed on March 21, 2024.
[7]
Nba website. “https://www.nba.com/”. Accessed on March 25, 2024.
[8]
One of my favorite nba offensive concepts. “https://www.youtube.com/watch?v=_wA4Fpzx08s”. Accessed on March 21, 2024.
[9]
Second spectrum. “https://www.secondspectrum.com/”. Accessed on March 25, 2024.
[10]
Sportvu camera system in nba. “https://www.statsperform.com/team-performance/basketball/optical-tracking/”. Accessed on March 25, 2024.
[11]
Statmuse. “https://www.statmuse.com/”. Accessed on March 14, 2024.
[12]
Viz libero. “https://www.vizrt.com/products/viz-libero.”. Accessed on March 25, 2024.
[13]
G. Altavilla, G. Raiola et al., Global vision to understand the game situations in modern basketball. Journal of Physical Education and Sport, 14:493–496, 2014.
[14]
M. Chen and R. Bunescu. Changing the narrative perspective: From deictic to anaphoric point of view. Information Processing & Management, 58(4):102559, 2021.
[15]
W.-G. Chen, I. Spiridonova, J. Yang, J. Gao, and C. Li. Llava-interactive: An all-in-one demo for image chat, segmentation, generation and editing. arXiv preprint, 2023.
[16]
A. Choudhry, M. Sharma, P. Chundury, T. Kapler, D. W. Gray, N. Ramakrishnan, and N. Elmqvist. Once upon a time in visualization: Understanding the use of textual narratives for causality. IEEE Transactions on Visualization and Computer Graphics, 27(2):1332–1342, 2020.
[17]
X. Chu, X. Xie, S. Ye, H. Lu, H. Xiao, Z. Yuan, C. Zhu-Tian, H. Zhang, and Y. Wu. TIVEE: Visual Exploration and Explanation of Badminton Tactics in Immersive Visualizations. IEEE Transactions on Visualization and Computer Graphics, PP: 1–1, 2021.
[18]
J. Courel-Ibáñez, A. P. McRobert, E. Ortega Toro, and D. Cárdenas Vélez. Inside game effectiveness in nba basketball: Analysis of collective interactions. Kinesiology, 50(2.):218–227, 2018.
[19]
J. Courel-Ibáñez, A. P. McRobert, E. O. Toro, and D. C. Vélez. Collective behaviour in basketball: a systematic review. International Journal of Performance Analysis in Sport, 17(1–2):44–64, 2017.6.
[20]
A. C. A.M. de Faria, F. d. C. Bastos, J. V. N.A. da Silva, V. L. Fabris, V. d. S. Uchoa, D. G. d.A. Neto, and C. F. G.d. Santos. Visual question answering: A survey on techniques and common trends in recent literature. arXiv preprint, 2023.
[21]
C. A. Dietrich, D. Koop, H. T. Vo, and C. T. Silva. Baseball4d: A tool for baseball game reconstruction & visualization. 2014 IEEE Conference on Visual Analytics Science and Technology (VAST), pp. 23–32, 2014.
[22]
Y. Fu and J. T. Stasko. Hoopinsight: Analyzing and comparing basketball shooting performance through visualization. IEEE Transactions on Visualization and Computer Graphics, 30:858–868, 2023.
[23]
N. Gershon and W. Page. What storytelling can do for information visualization. Communications of the ACM, 44(8):31–37, 2001.
Digital Library
[24]
M. C. Green. Transportation into narrative worlds: The role of prior knowledge and perceived realism. Discourse processes, 38(2):247–266, 2004.
[25]
W. Javed and N. Elmqvist. Exploring the design space of composite visualization. In 2012 IEEE pacific visualization symposium, pp. 1–8. IEEE, 2012.
Digital Library
[26]
I. S. Kohli. On optimal offensive strategies in basketball. arXiv preprint, 2015.
[27]
M.-J. Kraak. The space-time cube revisited from a geovisualization perspective. In Proc. 21st international cartographic conference, pp. 1988–1996. Citeseer, 2003.
[28]
S. Kriglstein, M. Pohl, and M. Smuc. Pep up your time machine: Recommendations for the design of information visualizations of time-dependent data. Handbook of human centric visualization, pp. 203–225, 2014.
[29]
P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Ktittier, M. Lewis, W.-t. Yih, T. Rocktäschel et al., Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing Systems, 33:9459–9474, 2020.
[30]
T. Lin, A. Aouididi, C. Zhu-Tian, J. Beyer, H. Pfister, and J.-H. Wang. VIRD: Immersive Match Video Analysis for High-Performance Badminton Coaching. IEEE Transactions on Visualization and Computer Graphics, 30:458–468, 2023.
[31]
T. Lin, C. Zhu-Tian, Y. Yang, D. Chiappalupi, J. Beyer, and H. Pfister. The quest for omnioculars: Embedded visualization for augmenting basket-ball game viewing experiences. IEEE transactions on visualization and computer graphics, 29(1):962–971, 2022.
[32]
H. Liu, C. Li, Q. Wu, and Y. J. Lee. Visual instruction tuning. Advances in neural information processing systems, 36, 2024.
[33]
A. G. Losada, R. Therón, and A. Benito. Bkviz: A basketball visual analysis tool. IEEE Computer Graphics and Applications, 36:58–68, 2016.
Digital Library
[34]
E. Mayr and F. Windhager. Once upon a spacetime: Visual storytelling in cognitive and geotemporal information spaces. ISPRS International Journal of Geo-Information, 7(3):96, 2018.
[35]
A. McIntyre, J. Brooks, J. Guttag, and J. Wiens. Recognizing and analyzing ball screen defense in the nba. In Proceedings of the MIT sloan sports analytics conference, Boston, MA, USA, pp. 11–12, 2016.
[36]
D. S. Miall and D. Kuiken. Shifting perspectives: Readers' feelings and literary response. New perspectives on narrative perspective, pp. 289–301, 2001.
[37]
M. Mulcahy and B. Gouldthorp. Positioning the reader: the effect of narrative point-of-view and familiarity of experience on situation model construction. Language and Cognition, 8(1):96–123, 2016.
[38]
H. L. O'Brien and E. G. Toms. The development and evaluation of a survey to measure user engagement. Journal of the American Society for Information Science and Technology, 61(1):50–69, 2010.
Digital Library
[39]
J. S. Park, J. O'Brien, C. J. Cai, M. R. Morris, P. Liang, and M. S. Bernstein. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, pp. 1–22, 2023.
Digital Library
[40]
C. Perin, R. Vuillemot, and J.-D. Fekete. Soccerstories: A kick-off for visual soccer analysis. IEEE Transactions on Visualization and Computer Graphics, 19:2506–2515, 2013.
Digital Library
[41]
J. C. Roberts. Exploratory visualization with multiple linked views. In Exploring geovisualization, pp. 159–180. Elsevier, 2005.
[42]
S. Salvador and P. Chan. Toward accurate dynamic time warping in linear time and space. Intelligent Data Analysis, 11(5):561–580, 2007.
Digital Library
[43]
W. Schnotz. An integrated model of text and picture comprehension. The Cambridge handbook of multimedia learning, 49(2005):69, 2005.
[44]
K. Schröder, W. Eberhardt, P. Belavadi, B. Ajdadilish, N. van Haften, E. Overes, T. Brouns, and A. C. Valdez. Telling stories with data-a systematic review. arXiv preprint, 2023.
[45]
E. Segel and J. Heer. Narrative visualization: Telling stories with data. IEEE transactions on visualization and computer graphics, 16(6):1139–1148, 2010.
Digital Library
[46]
L. Shen, Y. Zhang, H. Zhang, and Y. Wang. Data player: Automatic generation of data videos with narration-animation interplay. IEEE Transactions on Visualization and Computer Graphics, 2023.
[47]
A. Sicilia, K. Pelechrinis, and K. Goldsberry. Deephoops: Evaluating micro-actions in basketball using deep feature representations of spatio-temporal data. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 2096–2104, 2019.
[48]
A. Singh. Optimizing performance in basketball: A game-theoretic approach to shot percentage distribution in a team. arXiv e-prints, pp. arXiv-2310, 2023.
[49]
B. Skinner and M. Goldman. Optimal strategy in basketball. In Handbook of statistical methods and analyses in sports, pp. 245–260. Chapman and Hall/CRC, 2017.
[50]
M. Stein, H. Janetzko, A. Lamprecht, T. Breitkreutz, P. Zimmermann, B. Goldlücke, T. Schreck, G. L. Andrienko, M. Grossniklaus, and D. A. Keim. Bring it to the pitch: Combining video and movement data to enhance team sport analysis. IEEE Transactions on Visualization and Computer Graphics, 24:13–22, 2018.
[51]
N. Sultanum and A. Srinivasan. Datatales: Investigating the use of large language models for authoring data-driven articles. In 2023 IEEE Visualization and Visual Analytics (VIS), pp. 231–235. IEEE, 2023.
[52]
C. Tian, V. De Silva, M. Caine, and S. Swanson. Use of machine learning to automate the identification of basketball strategies using whole team player tracking data. Applied Sciences, 10(1):24, 2019.
[53]
T.-Y. Tsai, Y.-Y. Lin, H.-Y. M. Liao, and S.-K. Jeng. Recognizing offensive tactics in broadcast basketball videos via key player detection. In 2017 IEEE International Conference on Image Processing (ICIP), pp. 880–884. IEEE, 2017.
Digital Library
[54]
B. Tversky, J. B. Morrison, and M. Betrancourt. Animation: can it facilitate? International journal of human-computer studies, 57(4):247–262, 2002.
[55]
Unkown. Glossary of basketball terms. “https://en.wikipedia.arg/wiki/Glossary_of_basketball_terms”, Oct. 2010. Accessed on March 14, 2024.
[56]
J. Wang, J. Wu, A. Cao, Z. Zhou, H. Zhang, and Y. Wu. Tac-miner: Visual tactic mining for multiple table tennis matches. IEEE Transactions on Visualization and Computer Graphics, 27:2770–2782, 2021.
[57]
J. Wang, K. Zhao, D. Deng, A. Cao, X. Xie, Z. Zhou, H. Zhang, and Y. Wu. Tac-simur: Tactic-based simulative visual analytics of table tennis. IEEE Transactions on Visualization and Computer Graphics, 26:407–417, 2020.
[58]
W. Willett, Y. Jansen, and P. Dragicevic. Embedded data representations. IEEE transactions on visualization and computer graphics, 23(1):461–470, 2016.
Digital Library
[59]
Y. Wu, D. Deng, X. Xie, M. He, J. Xu, H. Zhang, H. Zhang, and Y. Wu. Obtracker: Visual analytics of off-ball movements in basketball. IEEE Transactions on Visualization and Computer Graphics, 29:929–939, 2022.
[60]
L. Yao, R. Vuillemot, A. Bezerianos, and P. Isenberg. Designing for visualization in motion: Embedding visualizations in swimming videos. IEEE Transactions on Visualization and Computer Graphics, 30:1821–1836, 2023.2.
[61]
S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao. React: Synergizing reasoning and acting in language models. arXiv preprint, 2022.
[62]
S. Zhang, P. Sun, S. Chen, M. Xiao, W. Shao, W. Zhang, K. Chen, and P. Luo. Gpt4roi: Instruction tuning large language model on region-of-interest. arXiv preprint, 2023.
[63]
Z. Zhao, R. Marr, and N. Elmqvist. Data comics: Sequential art for data-driven storytelling. tech. report, 2015.
[64]
Q. Zhi, S. Lin, P. T. Sukumar, and R. A. Metoyer. Gameviews: Understanding and supporting data-driven sports storytelling. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, 2019.
[65]
Q. Zhi and R. A. Metoyer. Gamebot: A visualization-augmented chatbot for sports game. Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems, 2020.
Digital Library
[66]
C. Zhu-Tian, Q. Yang, J. Shan, T. Lin, J. Beyer, H. Xia, and H. Pfister. iBall: Augmenting Basketball Videos with Gaze-Moderated Embedded Visualizations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pp. 1–18, 2023.
[67]
C. Zhu-Tian, Q. Yang, X. Xie, J. Beyer, H. Xia, Y. Wu, and H. Pfister. Sporthesia: Augmenting sports videos using natural language. arXiv e-prints, pp. arXiv-2209, 2022.
[68]
C. Zhu-Tian, S. Ye, X. Chu, H. Xia, H. Zhang, H. Qu, and Y. Wu. Augmenting Sports Videos with VisCommentator. IEEE Transactions on Visualization and Computer Graphics, PP:1–1, 2021.
Index Terms
Sportify: Question Answering with Embedded Visualizations and Personified Narratives for Sports Video
Applied computing
Arts and humanities
Information systems
Information systems applications
Multimedia information systems
Index terms have been assigned to the content through auto-classification.
Recommendations
- iBall: Augmenting Basketball Videos with Gaze-moderated Embedded Visualizations
CHI '23: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems
We present iBall, a basketball video-watching system that leverages gaze-moderated embedded visualizations to facilitate game understanding and engagement of casual fans. Video broadcasting and online video platforms make watching basketball games ...
Read More
- Rules, gameplay, and narratives in video games
An explanatory study is conducted to examine the different kinds of rules in video games. Two layers of game, the abstract and the narrative layer, are explored to unify the study of gameplay and narratives. Derivatives of paidea rules and ludus rules ...
Read More
- e-Sports: Playing just for fun or playing to satisfy life goals?
Highlights
- Clan leaders significantly differed in life goal power from not members of any clan.
Abstract Purpose
e-Sports is an area of the game scene, in which computer game players specialize in a specific game, form game teams (clans), compete together in tournaments and meet at so-called LAN (Local Area Network) parties. ...
Read More
Comments
Information & Contributors
Information
Published In
IEEE Transactions on Visualization and Computer Graphics Volume 31, Issue 1
Jan. 2025
1276 pages
Issue’s Table of Contents
1077-2626 © 2024 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.
Publisher
IEEE Educational Activities Department
United States
Publication History
Published: 10 September 2024
Qualifiers
- Research-article
Contributors
Other Metrics
View Article Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
Total Citations
Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 30 Nov 2024
Other Metrics
View Author Metrics
Citations
View Options
View options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in
Full Access
Get this Publication
Media
Figures
Other
Tables
