Horace's BlogArt geek2025-12-01T03:31:10.997Zhttps://www.lumenghe.com/Horace LuHexoGoogle AI Studio Vibe Coding 小记https://www.lumenghe.com/2025/11/19/testing-google-ai-studio/2025-11-19T10:33:42.000Z2025-12-01T03:31:10.997Z
20251022
试了试 Google AI Studio 新的 build 功能，30分钟 vibe coding 成果：
这份契约的核心，并非关乎旋律或节奏，而是关于音乐的 使命。他在开场白中明确宣告：“呢啲歌系讲生活、讲爱，所有好的坏的” (These songs are about life, about love, all the good and the bad) 。这一宣告至关重要，它界定了方大同的创作疆域：他的音乐将是一个容器，忠实地承载生活与爱的二元性——“好的”与“坏的”并存，光明与阴影共生。
紧接着，他进一步阐明了这份契约的 哲学功能。音乐是关于“The things change us… The things make should we are (能让我们认识自己)” (The things change us… The things make should we are [Let us know ourselves]) 。因此，“Soulboy”不仅是一个风格的命名，更是一种哲学的立场。他的“灵魂乐”，是一种旨在促进自我认知（“认识自己”）与内在转变（“改变我地”）的媒介。
然而，经过十余年的历练，其灵魂的道德感变得更为复杂和深刻。在 2016 年的《Ring Finger》中，冲突不再是外部的“你”，而是转为了 内部 的“我”。歌曲生动地描绘了面对诱惑（“That girl is a danger zone”）时的内心挣扎（“只是谈谈心 又没关系 动一点凡心 也没关系”） 。这是一种对人性弱点的坦诚。但灵魂的道德罗盘最终战胜了欲望：“忘了指环的承诺”、“美梦忽然惊醒 这代价我付不起”、“I think I gotta go back, I gotta go back” 。
在《NMW (No Matter What)》（2016）中，他写道：“I’m never gonna love again… 我只想用整个未来守护着你” 。这句“I’m never gonna love again”并非一句冲动的情话，而是其灵魂在经历了所有情感可能性之后得出的 认知结论。在体验了《三人游》的牺牲与《爱不来》的无奈后，灵魂选择了一种“不再搜寻”的终极锁定。
这种对“大同”理想的追求，在其后续的创作中不断演进和深化。2011 年的《Gotta Make a Change》中，他疾呼：“Let’s change the world… 每个城 每个省 每个国家 每个人记得这是我们的家” 。到了 2014 年的《Peace》，这种呼吁已经系统化为一种清晰的哲学理念：“Peace, love, livin’ in harmony / One world in unity, my brotha, no enemy” (和平、爱、生活在和谐中 / 同一个世界，团结一致，我的兄弟，没有敌人) 。
最终，在单曲《TWIOCAMIC》中，他揭示了这一理想的灵魂根源。歌曲标题“TWIOCAMIC”即是“The World Is One Country, Mankind Its Citizens”（世界是同一个国家，人类是它的公民）的缩写，这是巴哈伊信仰（Baha’i Faith）的核心教义之一。歌词“Not you, not me, not them, just us / One planet, one people for true justice” (不是你，不是我，不是他们，只有我们 / 一个星球，一个民族，为了真正的正义) 以及“Two wings of one bird is the greatest analogy” (一体双翼是最好的比喻，暗喻两性平等) 均是其精神信仰的音乐转译。这充分证明，方大同的“和平”与“大同”并非空泛的口号，而是其灵魂深处坚定的 精神实践。
同时，音乐也是他对抗现代性焦虑的最终解药。在单曲《Favorite Stuff》中，他唱道：“Don’t give me drugs, just all the things that I love… It’s my favorite stuff” 。面对《烦》中“世界很乱”的局面，他拒绝麻醉品，而是选择音乐作为灵魂的“最爱”与最终的精神寄托。
风、树林与时代精神（Wind, Woods & Zeitgeist）：风象征着“神秘”，也代表林奇所说的“空气中的东西”（Thing in the Air），即时代精神（Zeitgeist），一种由现代媒体加速的、弥漫性的恐惧氛围。这种“风”通过电波（airwaves）传播，塑造了电视这团“火”的形态。树林中的树木，尤其是梧桐树（sycamore trees），象征着电线杆，它们是承载电流（火）的“树枝”。
第三层，也是其理论的核心，是 元叙事或媒介隐喻层面。Twin Perfect 主张，《双峰》从创作初衷上就是一部“关于电视的剧”（a show about the concept of television），林奇利用剧情隐喻地批判了电视产业的陈俗与堕落，描绘了真实创意如何逐渐滑向贫乏娱乐的过程。例如，剧中的戏中戏《爱之邀约》（Invitation to Love）便是对肥皂剧俗套的直接映射，提示观众该剧正在反思电视本身。Twin Perfect 认为，第一季和第二季前半段在林奇主导下充满了对传统电视的讽喻和批评，一方面呈现警探调查与情感纠葛等类型化情节，另一方面通过夸张的情节和超现实插入（如巨人谜语、红房梦境）来揭示电视叙事的模式和空洞。而当剧集因电视网压力而透露凶手、走向庸俗化时（第二季中段林奇暂离期间），林奇有意将《双峰》自身的质量下滑也纳入了隐喻，以展示电视媒介如何“腐蚀”艺术创作。第三季《双峰：回归》更被视为林奇对现代电视文化的全面反思，如怀旧的诱惑与危险、观众期望与叙事颠覆等，都在其中得到自指式呈现。剧中多处情节被解读为在打破第四面墙，揭示角色逐渐意识到自身“不是真实存在”的虚构性：劳拉在红房间哭泣时脸上闪烁的电视光、库珀与黛安驾车进入新现实后的身份错位，以及最终劳拉回到故居却发现一对现实中的陌生夫妻（由真实房主扮演）居住其中而发出的震惊尖叫，这些都被视为角色意识到自己“仅存于电视剧中”的证据，直接点破了作品的元叙事本质。
时代病症——可消费的电视暴力（Consumable TV Violence）。林奇认为，当时的主流电视（尤其是罪案剧）存在一种“病态”。谋杀案在一小时内被解决，受害者成为一个被消费后即遭遗忘的符号。观众只关心“谁是凶手”的智力游戏，而忽略了暴力带来的真实痛苦、悲伤和对社区的持久影响。这种剔除了悲伤、使其变得“有趣”的暴力，就是“可消费的电视暴力”。它让观众获得廉价的“终结感”，然后迅速转移到下一个消费品上。
Twin Perfect 的视频发布后在粉丝圈和媒体间引起巨大反响。Rosseter（频道主）并未止步于主视频，而是在不同平台上进一步阐释和捍卫他的理论。针对主视频发布后的争议，他先后制作了两部补充视频：一部是《Twin Peaks Actually Explained: EVEN MORE Evidence》（更多证据），另一部是《Twin Peaks Actually Explained: Responding to the Critics》（回应批评）。
在 Twin Perfect 的理论走红后，一些媒体和播客也邀请 Rosseter 讨论他的观点。2019 年 10 月，他接受了《Twin Peaks The Return: A Season Three Podcast》播客的专访。在这期播客中，主持人提到他的 4.5 小时视频“引发的解读和批评几乎是第三季结束以来最多的”。Rosseter 在采访中承认他的见解有争议，但也获得许多观众的赞赏。他半开玩笑地表示视频标题带有吸引眼球的考虑（SEO 效果达成后或许会改名），但依然坚持视频中的具体结论。这次访谈让他有机会更从容地阐述理论背后的灵感和逻辑。据节目的文字介绍，他的分析“无论你是否认同，都一定会让你想再看一遍《双峰》”。这种自信也反映在 Rosseter 对自己理论价值的看法上：他相信自己为《双峰》谜团贡献了新的视角，哪怕无法说服所有人，至少能让粉丝重新审视剧中线索。
Summary — Color constitutes another step in the steady advancement of the motion picture toward realism, the same principles of color, tone, and composition applying to the motion picture as to the art of painting. In order fully to appreciate the color picture, a “color consciousness” must be adopted, the lack of which is tantamount in a degree to color blindness.
Monotony is the enemy of interest, a fact that argues for the color picture; but a superabundance of color is unnatural. Psychologically, colors fall into the “warm” and “cool” groups, and each color and shade has its psychological implications: red — danger, blood, life, heat; green — nature, outdoors, freedom, freshness; etc. To build up personalities and to harmonize emotions and situations, these principles must apply, even to the extent of “color juxtaposition,” or the psychological relation of the various colors to each other. For example, of two adjacent or contiguous colors, each tends to “throw” the other toward its complement, considerably affecting the emphasis or import of the color.
On the walls of the cave in Altamira in Spain are found paintings, boldly sketched in three colors by Paleolithic man some fifty thousand years ago. These prehistoric paintings are quite artfully executed, and show that the artist possessed a fine sense of color and a desire to indicate motion as well as form. Various animals are depicted with the use of a red clay, an ochre earth, and a black pigment. One picture shows a wild boar in a standing position. In another picture, nearby, to show the same animal in a gallop, two sets of legs have been used. This ingenious method of showing action indicates the inherent desire of the artist to show motion in color. This ambition has come down through the intervening years to the present day. Now we see the culmination of that idea — motion pictures in color.
From a technical standpoint, motion pictures have been steadily tending toward more complete realism. In the early days, pictures were a mere mechanical process of imprinting light upon film and projecting that result upon a screen. Then came the perfection of detail — more accurate sets and costumes — more perfect photography. The advent of sound brought increased realism through the auditory sense. The last step — color, with the addition of the chromatic sensations, completed the process. Now motion pictures are able to duplicate faithfully all the auditory and visual sensations.
This enhanced realism enables us to portray life and nature as it really is, and in this respect we have made definite strides forward. A motion picture, however, will be merely an accurate record of certain events unless we guide this realism into the realms of art. To accomplish this it becomes necessary to augment the mechanical processes with the inspirational work of the artist. It is not enough that we put a perfect record upon the screen. That record must be molded according to the basic principles of art.
The principles of color, tone, and composition make painting a fine art. The same principles will make a colored motion picture a work of art. The precision and detail of Holbein and Bougereau, the light effects of Rembrandt, the atmosphere and arrangements of Goya, the color of Velasquez, the brilliant sunlight of Sorolla, the mysterious shadows of Innes — all these artistic qualities can eventually be incorporated into motion pictures through the medium of color. The design and colors of sets, costumes, drapes, and furnishings must be planned and selected just as an artist would choose the colors from his pallette and apply them to the proper portions of his painting.
In order to apply the laws of art properly in relation to color, we must first develop a color sense — in other words, we must become “color conscious.” We must study color harmony, the appropriateness of color to certain situations, the appeal of color to the emotions. Above all, we must take more interest in the colorful beauties that lie about us — the iridescent brilliance of the butterfly’s wing, the subtle tones of a field of grain, the violet shadows of the desert, the sunset’s reflection in the ocean. By such observation and study we develop a sense of color appreciation and train our eye to notice an infinite variety of hues.
Serious cases of color blindness are comparatively rare; yet, because the average person is not trained in color appreciation, a decided lack of color consciousness is not at all uncommon. In order to appreciate operatic or classical music, people study music appreciation. Color appreciation, as a study, is almost entirely neglected, although color plays a most important and continuous part in ourlives. The average person listens to music for only a short portion of the time, but every moment of the day he looks upon some form of color.
In the study of color appreciation we have two classes of objects. On the one hand, we have Nature, with its flowers, skies, trees, etc.; on the other hand, we have man-made objects of all kinds, including art pictures. In the first class the color is already created, and it remains for us only to enjoy and appreciate. In the second class we can exercise a certain amount of selectivity. Because of the general lack of color knowledge, that selectivity is not always tempered with wisdom. If the color schemes of natural objects were used as guides, less flagrant mistakes in color would occur. The use of black and white, however, to the complete exclusion of all color, is decidedly not in keeping with Nature’s rules.
Natural colors and lights do not tax the eye nearly as much as man-made colors and artificial lights. Even when Nature indulges in a riot of beautiful colors, there are subtle harmonies which justify those colors. These harmonies are often overlooked by the casual observer. The most brilliant flower has leaves and stem of just the right hue to accompany or complement its gay color.
As we grow in the understanding of color and its uses, we find that our color appreciation develops simultaneously. All the better things in life require a color consciousness for their fullest appreciation and enjoyment.
The eye is the organ of perception. The impulses of light received by the retina are transferred over the optic nerve path to the brain, and we become conscious of light and dark, motion, form, and color. Vision is a sense of ancient lineage and of early development in the individual life. Its characteristic is the clearness and precision of the data it furnishes the mind. Compared to sight, the other senses are dull and groping. It is the sense by which we receive the greatest number of stimuli from the world about us. It is the sense which most frequently affects the nervous system, dominates the attention, and stimulates the mind.
It is a psychological fact that the nervous system experiences a shock when it is forced to adapt itself to any degree of unnaturalness in the reception of external stimuli. The auditory sense would be unpleasantly affected by hearing an actor upon the screen speak his lines in a monotone. The mind would strive to supply the missing inflections. The same is true, but to a greater degree, of the visual sense. A super-abundance of color is unnatural, and has a most unpleasant effect not only upon the eye itself, but upon the mind as well. On the other hand, the complete absence of color is unnatural. The mind strives to supply the missing chromatic sensations, just as it seeks to add the missing inflections to the actor’s voice. The monotony of black, gray, and white in comparison with color is an acknowledged fact. It is almost a psychological axiom that monotony is the enemy of interest. In other words, that which is monotonous will not hold our attention as well as that which shows more variety. Obviously, it is important that the eye be not assailed with glaring color combinations, nor by the indiscriminate use of black and white. Again taking our cue from Nature, we find that colors and neutrals augment each other. The judicious use of neutrals proves an excellent foil for color, and lends power and interest to the touches of color in a scene. The presence of neutrals in our composition adds interest, variety, and charm to our colors. On the other hand, the presence of color in our picture gives added force to the neutrals, emphasizing the severity of black, the gloominess of gray, the purity of white.
From a broader point of view, the psychology of color is of immense value to a director. His prime motive is to direct and control the thoughts and emotions of his audience. The director strives to indicate a fuller significance than is specifically shown by the action and dialog. If he can direct the theatergoer’s imagination and interest, he has fulfilled his mission. The psychology of color is all-important in this respect, and we shall now show the manner in which certain colors upon the screen will give rise to certain emotions in the audience.
We have found that by the understanding use of color we can subtly convey dramatic moods and impressions to the audience, making them more receptive to whatever emotional effect the scenes, action, and dialog may convey. Just as every scene has some definite dramatic mood — some definite emotional response which it seeks to arouse within the minds of the audience — so, too, has each scene, each type of action, its definitely indicated color which harmonizes with that emotion.
The usual reaction of a color upon a normal person has been definitely determined. Colors fall into two general groups. The first group is the “warm,” and the second the “cool” colors. Red, orange, and yellow are called the warm or advancing colors. They call forth sensations of excitement, activity, and heat. In contrast, green, blue, and violet are the cool or retiring colors. They suggest rest, ease, coolness. Grouping the colors in another manner we find that colors mixed with white indicate youth, gaiety, informality. Colors mixed with gray suggest subtlety, refinement, charm. When mixed with black, colors show strength, seriousness, dignity, but sometimes represent the baser emotions of life.
As to the use of a single color alone, each hue has its particular associations. For example, red recalls to mind a feeling of danger, a warning. It also suggests blood, life, and love. It is materialistic, stimulating. It suffuses the face of anger, it led the Roman soldiers into battle. Different shades of red can suggest various phases of life, such as love, happiness, physical strength, wine, passion, power, excitement, anger, turmoil, tragedy, cruelty, revenge, war, sin, and shame. These are all different, yet in certain respects they are the same. Red may be the color of the revolutionist’s flag, and streets may run red with the blood of rioters, yet red may be used in a church ritual for Pentecost as a symbol of sacrifice. Whether blood is spilled upon the battlefield in an approved cause or whether it drips from the assassin’s dagger, blood still runs red. The introduction of another color with red can suggest the motive for a crime whether it be jealousy, fanaticism, revenge, patriotism, or religious sacrifice. Love gently warms the blood. The delicacy or strength of the shade of red will suggest the type of love. By introducing the colors of licentiousness, deceit, selfish ambition, or passion, it will be possible to classify the type of love portrayed with considerable accuracy.
Proceeding to the other colors, orange is bright and enlivening; it suggests energy, action.
橙色明亮且富有活力，象征能量与行动力。
Yellow and gold symbolize wisdom, light, fruition, harvest, reward, riches, gaiety; but yellow also symbolizes deceit, jealousy, inconstancy in its darker shades, and particularly when it is tinged with green.
Green immediately recalls the garb of Nature, the outdoors, freedom. It also suggests freshness, growth, vigor.
绿色立即让人联想到大自然、户外环境与自由，同时也暗示清新、生长与活力。
Dark green, blue, violet, and indigo are cooling, quiet colors. They are tranquil and passive. They do not suggest activity, as do the reds and orange. Blue is suggestive of truth (“true blue”), calm, serenity, hope, science, also cold steel, melancholy (we have the expression “blue as indigo”).
Purple is a color which does not occur in the spectrum. It is a combination of warm red and cool blue. It will be aggressive and vital if the red predominates, or dignified and quiet if the blue overbalances the red. Purple denotes solemnity, royalty, also pomp and vanity.
Magenta is the combination of purple and red. It is very distinctly materialistic. It is showy, arrogant, and vain.
洋红色由紫色与红色混合而成，带有明显的物质主义色彩，表现出张扬、傲慢与虚荣。
The neutrals, white, gray, and black, while theoretically not in the category of colors, also stimulate very definite emotional responses. Black is no color, but absorption of all color. It has a distinctly negative and destructive aspect. Black instinctively recalls night, fear, darkness, crime. It suggests funerals, mourning. It is impenetrable, comfortless, secretive. It flies at the masthead of the pirate’s ship. Our language is replete with references to this frightful power of black — black art, black despair, black-guard, blackmail, black hand, the black hole of Calcutta, black death (the devastating plague of medieval Europe), black list, black-hearted, etc.
白色、灰色与黑色虽然在理论上并不属于色彩范畴，但它们同样能够引发强烈的情绪反应。黑色本质上并非一种颜色，而是所有颜色的吸收。它具有明显的负面与破坏性特征，直觉上让人联想到黑夜、恐惧、黑暗与犯罪。黑色暗示葬礼、哀悼，具有深不可测、令人不安且充满隐秘感的特性。它飘扬在海盗船的桅杆上，象征死亡与威胁。语言中也充满了对黑色负面力量的隐喻，例如“黑魔法”（black art）、“黑暗绝望”（black despair）、“恶棍”（blackguard）、“勒索”（blackmail）、“黑手党”（black hand）、“加尔各答黑牢”（the black hole of Calcutta）、“黑死病”（black death，中世纪欧洲的毁灭性瘟疫）、“黑名单”（blacklist）以及“黑心”（black-hearted）等。
Even the poets recognized this symbolism. Shelly, in his dramatic Alaster tells how,
即使是诗人也认识到了这种色彩象征意义。诗人雪莱在其戏剧作品《阿拉斯托耳》（Alastor）中写道：
“ I have made my bed In charnels and on coffins, where black death Keeps record of the trophies won — “ “我已在尸骨遍布的墓穴和棺木之上安身， 在黑死病记录其战利品的地方——”
The poet Keats, in The Prisoner of Chilian, says,
诗人济慈（译者注：应为拜伦勋爵）在《西庸的囚徒》（The Prisoner of Chillon）中写道：
“ I, only, stirred in this black spot, I, only, drew the accursed breath of dungeon-dew.” “唯有我，在这片黑暗之地微微颤动， 唯有我，呼吸着这可诅咒的地牢湿气。”
We are speaking a potent language to our audience when we make use of black.
当我们在电影中运用黑色时，实际上是在向观众传递一种极具力量的视觉语言。
Gray suggests gray skies and rain. It is gloomy, dreary, and represents solemnity and maturity. From its complete neutrality and lack of any color or distinctiveness, it represents mediocrity, indecisiveness, inaction, vagueness.
White reflects the greatest amount of light, it emanates a luminosity which symbolizes spirit. White represents purity, cleanliness, peace, marriage. Its introduction into a color sublimates that color. For example, the red of love becomes more refined and idealistic as white transforms the red to pink. White uplifts and ennobles, while black lowers and renders more base and evil any color. To the degree in which colors are lightened or darkened will the qualities that the color exemplifies be altered.
Thus we see that all the colors in the spectrum speak their particular language. The flush of anger, the vigor of a sun-tanned skin, the richness of gold velvet, the violet mystery of distant mountains, the serenity of blue sky — these colors alone speak with more eloquence than could be described by words.
The modification of a positive color by the introduction of another hue modifies the mental reaction to the degree of the intensity of that hue which is introduced. For example, a positive blue is a cool color, but to the extent in which a red hue is introduced, the coolness of the blue will be altered by the warmth of red. However, these complexities do not alter the basic principles of color or the general reactions which we have outlined.
In the preparation of a picture we read the script and prepare a color chart for the entire production, each scene, sequence, set, and character being considered. This chart may be compared to a musical score, and amplifies the picture in a similar manner. The preparation of this chart calls for careful and judicious work. Subtle effects of beauty and feeling are not attained through haphazard methods, but through application of the rules of art and the physical laws of light and color in relation to literary laws and story values. In the first place, this chart must be in absolute accord with the story action. Again, it must consider the art, principles of unity, color harmony, and contrast. Again, it must consider the practical limitations of motion picture production and photography. The art director, however, in handling a color picture, must be forever mindful that the human eye is many times more sensitive than the photographic emulsion and many times greater in scope than any process of reproduction. Therefore, he must be able to translate his colors in terms of the process.
When we receive the script for a new film, we carefully analyze each sequence and scene to ascertain what dominant mood or emotion is to be expressed. When this is decided, we plan to use the appropriate color or set of colors which will suggest that mood, thus actually fitting the color to the scene and augmenting its dramatic value.
We plan the colors of the actor’s costumes with especial care. Whenever possible, we prefer to clothe the actor in colors that build up his or her screen personality. In a picture which we recently completed, two young girls play the parts of sisters. One is vivacious, affectionate, and gay. The other is studious, quiet, and reserved. For the first we planned costumes of pink, red, warm browns, tan, and orange; for the second, blue, green, black, and grey. In this way the colors were kept in unison with their film characters.
One very important phase of making color pictures is the necessity of obtaining distinct color separation. The term “color separation” means that when one color is placed in front of or beside another color, there must be enough difference in their hues to separate one from the other photographically. For example, there must be enough difference in the colors of an actor’s face or costume and the walls of the set to make him stand out from the colors back of him; otherwise, he will blend into the background and become indistinguishable, as does a polar bear in the snow. If the colors are properly handled, it is possible to make it appear as though the actors were actually standing there in person, thus creating the illusion of the third dimension. Because of the general warm glow of flesh tints, we usually introduce the cooler tones into the backgrounds; but, if we find it advantageous to use warmer tones in the set, we handle the lighting so that the particular section in back of the actor is left in shadow. This gives a cool contrast to the faces, even though we have a general feeling of warmth in the room. When there are a number of players, all wearing differently colored costumes, it is necessary to disregard those playing relatively unimportant parts, and make the background in contrast to those whose action is most significant to this particular scene.
It is important that the sets have interest and variety. They must not be flat. When the sets have depth it is much easier to introduce interesting shadows and colored lights for special effects. Unless the dramatic aspect dictates to the contrary, it is desirable to have all the colors in any one scene harmonious. Otherwise, we strike an unpleasant, discordant note.
A point to be considered in set dressing depends upon one of the rules of composition in art. The law of emphasis states in part that nothing of relative unimportance in a picture shall be emphasized. If, for example, a bright red ornament were shown behind an actor’s head, the bright color would detract from the character and action. Errors of this nature must be carefully avoided.
Color juxtaposition also plays a large part in the selection of colors for the screen. The effect of “color juxtaposition” is an apparent change of hue when different colors are placed one over the other, or side by side. If two cards, one orange, the other blue-green, are placed side by side, the orange will appear more red than it really is, the blue-green more blue. Each color tends “to throw” the other toward its complement. In other words, the complement of orange is blue; therefore, the orange makes the blue-green appear bluer. When any two colors are placed together, the first emphasizes in the second the characteristics which are lacking in the first.
It can readily be seen from this how exceedingly important it is to consider the movement in the scene in determining its color composition because the juxtaposition of colors is constantly changing due to this movement. Quite a different problem from that of an artist, who paints a still scene where the characters remain in their set places, and whose color values, therefore, are not subject to frequently changing contrast.
We must constantly practice color restraint. In the early two-color pictures, producers sometimes thought that because a process could reproduce color, they should flaunt vivid color continually before the eyes of the audience. This often led to unnatural and disastrous results, which experience is now largely eliminating.
The synthesis of all these factors entails many conferences with directors, art directors, writers, cameramen, designers and others. Technicolor color directors, cameramen, and technicians act in a consulting and advisory capacity to the various studio departments during both the preparation and the shooting of the picture.
Music, graphic art, and acting have now been united, and become one expression of more ultimate art. Now for the first time a perfect expression of the combined inspirations of producer, writer, artist, actor, and musician can be adequately presented to an audience. Color has touched the sound picture and it fairly lives.
Dodge College 的 Master Class Dinner 机制很值得学习：会抽选十几名学生与嘉宾在 career center 共进晚餐，有机会深入交流行业观点，而不只是映后泛泛而谈。餐桌前大部分是本科生，我与导师和其他几位老师坐在旁边。55 岁的 Delpy 如今是常居 LA 的导演/编剧，这次带来的是法国小镇乌克兰难民题材喜剧《Meet the Barbarians》。
几件趣事：聊到 Before Sunrise，她说剧本最早很男凝，是她向另两人建议，要么没有一夜情再也不见，要加一夜情结尾就得再约见。中间假装打电话的经典情节是她写的剧本，当时觉得只有真写出来导演才当回事，但最后她还是没挂名编剧。俩人 01 年拍完《半梦半醒的人生》后，觉得应该继续之前的故事，就开启了三人共同创作模式。学生问及是否重访过维也纳，她说维也纳拍夜戏体验非常累，每天只睡四个小时还很热，以及她不太喜欢奥地利人（笑）。
对演员来说，这种拍摄方式像话剧和电视剧的结合，既要面对观众表演，又要适应导演调度节奏，这意味着有几百号观众不仅在你演绎台词时会笑，在你 NG 时笑得更欢，想想还是颇有压力的。实际拍摄过程中，演员 NG 通常不会停机，而是直接从错误点继续表演，每个片段一般重拍 2 至 3 条，确保不同景别素材充足、表演衔接到位。现场见证几位经验丰富的主演在“保一条”时即兴发挥（ad-libbing），效果拔群，但整体仍在导演和剧本框架内，不受观众反馈影响。
对观众而言，清晰看到演员在镜头前演出、NG、重拍的全过程，其实打破了影视作品本身制造的幻象，在这种情境下还要全程保持亢奋也不容易。因此录制过程中会有一位剧组安排的资深喜剧演员（本场是 Alex Thomas）在观众席全程卖力单口相声加互动，指挥大家鼓掌欢呼，再配合机灵的 DJ 全程放歌，确保气氛持续热烈。当天现场观众一大半是黑人，DJ 放啥都能一起跟唱，还有好几位“素人好声音”自告奋勇前排献唱，而我基本都没听过，多少感受到一些与黑人社群的 culture gap（不过在放到 Kendrick Lamar 并听到主持老哥 diss Drake 时还是跟着笑出了声）。
法国化学家谢弗勒尔（Michel Eugène Chevreul）对“同时对比效应”的研究指出，当一种原色与其互补色相邻时，会使彼此更加鲜艳。很多艺术家从查尔斯·布朗（Charles Blanc）的著作《绘画艺术语法》（Grammaire des arts du dessin）了解到这种色貌现象，其中就包括梵·高推崇的画家欧仁·德拉克罗瓦（Eugène Delacroix）。在《鸢尾花》中，梵·高也受该效应启发，以紫色鸢尾花与黄色背景形成对比，使花朵更显生机勃勃。下图书中的一系列色轮进一步展示了同时对比效应的规律。
然而，随着时间推移，《鸢尾花》原本的紫色在当前展厅中呈现为蓝色，与梵·高信中的描述不符。Getty Center 研究人员采用X射线荧光光谱扫描（XRF scanning）等非侵入式技术分析画作的化学成分，并结合对当时可用颜料的研究，推断画家调配颜料的方式。
出处：Planetarian, 2017, 46(1) 标题：Filmmaking for Fulldome: Best Practices and Guidelines for Immersive Cinema (Part II) 作者：Ka Chun Yu, Dan Neafus, Ryan Wyatt 翻译：Horace Lu
（注：键盘快捷键“w”或左侧菜单右上角按钮，可切换文章列表视图与大纲视图）
[TOC]
In Part I of this paper, published in the December 2016 Planetarian (Yu et al. 2016), we discussed the critical differences between fulldome and traditional cinema. We described the pacing of fulldome films, which has its origins in how giant screen films are shot and edited. We described a theoretical framework for how to think of fulldome as a form of immersive cinema, after making the fundamental assumption that the audience feels they are “part of the action,” experiencing firsthand what is shown on-screen. We described how some of the basic grammar of film can be re-thought given this assumption.
本文的第一部分发表于 2016 年 12 月号的《Planetarian》杂志 (Yu et al. 2016)，在其中我们探讨了全景球幕电影与传统电影的根本差异。我们阐述了全景球幕电影的叙事节奏，这与巨幕电影的拍摄和剪辑方式息息相关。该论文基于一个基本的设想，即：观众感觉自己是“行动的一部分”，直接体验着银幕上所展示的一切。基于这个设想，我们构建了一个理论框架，将全景球幕电影视为一种沉浸式电影形式。我们还讨论了在这一设想下，如何重新构想电影的基本语法规则。
Keeping viewer attention 持续吸引观众注意力
Historically, filmmakers have used two methods to manipulate the viewer’s emotion, perception, and attention. The first is mise-en-scène, or the organization of everything inside the frame, including the arrangement of the actors, the set, lighting, and even camera motion. The second is through film editing, where shots are juxtaposed together in an order designed for a particular effect. We shall draw attention to the role of mise-en-scène later in the paper, but will focus on editing first.
For most of the 20th century, film footage was spliced and physically joined together into sequences, where individual shots instantaneously transition from one to the next, with this transition known as the “cut.” The style of cutting known as continuity editing developed in the 1910s with the work of Edwin Porter and D.W. Griffith (Bordwell & Thompson 2006). This style shows the action in spatially and temporally consistent ways from shot to shot, to make it easy for an audience to follow the narrative.
Another style of editing arose from the Soviet theory of montage, developing from filmmakers like Lev Kuleshov and Sergei Eisenstein. They believed that combining and assembling individual shots, even ones that have only weak spatial and temporal associations with each other, can synergistically give rise to complex ideas. Due to such influences, U.S. films in the silent era evolved to have rapid cutting, with average shot lengths down to 4-6 seconds (Bordwell 2006, p. 121). Both continuity and montage editing are employed today in traditional narrative cinema.
另一种剪辑方式发源于苏联的蒙太奇理论，由列夫·库列肖夫和谢尔盖·爱森斯坦等电影先驱提出，他们认为即使是空间和时间联系较弱的镜头，通过创造性的组合也能产生新的意义，激发深层次的思想。这种理念影响了美国无声电影时期的剪辑手法，使得平均镜头长度缩短至 4 到 6 秒，创造出快速剪辑的效果 (Bordwell 2006, p. 121)。至今，连贯性剪辑和蒙太奇剪辑仍然是传统叙事电影中不可或缺的技术。
Continuity editing in film has been successful because it results in clear storytelling that may be understood even by those who have little or no experience with the conventions of cinematic narratives (Schwan & Ildirar 2010). The effectiveness of continuity editing has been further confirmed in studies using eye tracking measurements that reveal that viewers watching scenes from professionally edited films and video sequences tend to cluster their gazes on the same part of the screen at the same time (Stelmach, Tam, & Hearty 1991; Goldstein, Woods, & Peli 2007; Mital et al. 2011). Sequences of moving images that were not edited together or edited sequences that were watched repeatedly do not have the same synchronous attention from viewers (Dorr et al. 2010).
连贯性剪辑之所以在电影创作中广受欢迎，是因为它能够讲述清晰易懂的故事，即便是对电影叙事手法不太熟悉或完全不熟悉的观众也能理解（Schwan & Ildirar 2010）。眼动追踪研究进一步证实了连贯性剪辑的有效性。研究发现，观众在观看经过专业人士剪辑的电影和视频片段时，往往会在同一时间将视线集中在银幕同一区域（Stelmach, Tam, & Hearty 1991; Goldstein, Woods, & Peli 2007; Mital et al. 2011）。相比之下，未经剪辑的视频片段，或经过剪辑但反复播放的视频片段，则无法以如此同步的方式吸引观众的注意力（Dorr et al. 2010）。
For a highly immersive frameless experience such as giant screen or fulldome, we do not expect the audience reaction to be identical to traditional cinema. In giant screen and fulldome filmmaking, the number of cuts are reduced and camera motions are slowed down in order to minimize motion sickness (Wollen 1993). As noted in Paper I (p. 29), audiences are immersed in their visual environments deeply enough that they feel the action is happening to them.
Because cuts are minimized (or left out altogether in a narrative journey mode), there can (and should) be ample opportunities for viewers to explore the scene. For extended shots lasting for more than a few seconds, there may be an initial shared point of interest, but as the shot continues without change, viewers’ attention will wander so their points of The safe spaces defined in Fig. 4 of Paper I focus will vary from individual to individual (Dorr et al. 2010; Mital et al. 2011).
在全景球幕电影中，为了让观众有更多机会深入探索场景，通常会尽量减少剪辑次数，甚至在“叙事旅程”模式中会完全避免剪辑。尽管观众最初可能会对某个场景产生共同的兴趣，但随着镜头长时间保持不变，他们的注意力可能会开始分散，导致每个人关注的焦点各有不同 (Dorr et al. 2010; Mital et al. 2011)。
To maintain story flow, the viewer’s attention cannot be allowed to wander indefinitely. A good director has to devise ways to redirect the audience’s focus. However, the viewer has immense freedom to look in different directions in fulldome, since the visible screen space (even within the safe spaces defined in Fig. 4 of Paper I) encompasses a significant fraction of the audience’s field of view. One way is to use audio cues, which can be highly directional if the theater is equipped with multiple speakers and three-dimensional sound spatialization software and hardware.
However, given the wide differences in audio experience between venues, a shareable audio standard for planetariums does not yet exist (Rodigast and Gaston 2010). We therefore focus on visual cues which translate easily between different venues because the domemaster is used as a common medium.
然而，由于不同场馆的音响效果存在显著差异，目前还没有一个统一的天文馆音响标准 (Rodigast and Gaston 2010)。因此，我们更倾向于使用视觉提示来引导观众，因为这些提示不依赖于特定的音响系统，其效果可以更容易地在不同的场馆之间传递，球幕母版作为共同媒介确保了视觉信息的一致性。
Visual cues for drawing attention 吸引注意力的视觉元素
Filmmakers have discovered the types of mise-en-scène cues that draw attention from viewers, which includes movement, faces, bright regions in the visual field, and features with different color. Researchers have been trying to not only substantiate these claims, but to understand how these effects work in the context of the scientific understanding of human perception and cognition.
Based on a review of the literature of some of this prior work, we identify the most important cues we believe can be used to direct attention in a fulldome film. This work complements ideas put forward by other practitioners in the field, notably the “domography” concept from the NSC Creative production team, which describes how to create effective fulldome storytelling with different types of shots, ways to emphasize content within the frame, and techniques for redirecting attention (Bradbury 2016).
Retreat to the center: In traditional cinema, the region of audience focus tends to be near the center of the frame (Goldstein, Woods, & Peli 2007; Brasel & Gips 2008). If there is a cut and there is nothing to sustain viewer attention elsewhere in the frame, the viewer’s eyes tend to return to the center (Tseng et al. 2009). When rapid cutting occurs, such as in movie trailers or music videos, the viewer’s focus also tends to migrate to the center (Tosi, Mecacci, and Pasquali 1997; Le Meur et al. 2007; Dorr et al. 2010; Mital et al. 2011).
在传统电影中，观众的焦点往往集中在画面中心 (Goldstein, Woods, & Peli 2007; Brasel & Gips 2008)。当镜头切换后，如果画面中没有其他元素吸引注意力，观众的视线倾向于回到中心。在电影预告片或音乐视频等剪辑节奏快的情境，观众的注意力同样倾向于集中到中心位置 (Tosi, Mecacci, and Pasquali 1997; Le Meur et al. 2007; Dorr et al. 2010; Mital et al. 2011)。
Even for static shots and regardless of the placement of content within the frame, there is evidence suggesting that attention is always biased towards the center (Tatler 2007; Tatler, Baddeley, & Gilchrist 2005). There is some evidence for a retreat to the center in immersive media in a study of a virtual reality ride, where viewers tend not to turn their heads much despite having the freedom to look in all directions (Pausch et al. 1996)[1]. We speculate that some form of this retreat to the center will also occur in fulldome films. In a film designed for unidirectional theaters, where the audience faces forward, the “default” viewpoint will be in the same general direction, so it is straightforward for a director to place visual content to be noticed by the audience.
Laboratory studies of the visual system show that motion is a powerful cue for capturing attention (Franconeri, Hollingworth, & Simons 2005). Motion in the cinematic frame leads the viewer to focus on the object in motion or on the part of the visual field where motion is occurring (Stelmach, Tam, & Hearty 1991; Mital et al. 2011). The greater the amount of motion and change in a video sequence, the more likely that different viewers’ attention will cluster toward the source of the motion (Stelmach, Tam, & Hearty 1991). In Treuting’s (2006) qualitative study using film clips, motion has been found to be more important for drawing attention than faces or bright regions inside the frame.
针对视觉系统的实验室研究表明，运动（motion）是吸引注意力的强大因素 (Franconeri, Hollingworth, & Simons 2005)。电影画面中的运动会使观众的焦点转移到运动物体或运动发生的视觉区域 (Stelmach, Tam, & Hearty 1991; Mital et al. 2011)。视频片段中的运动和变化越多，不同观众的注意力就越容易集中到运动源上 (Stelmach, Tam, & Hearty 1991)。Treuting (2006) 的定性研究发现，“运动”在吸引注意力方面的作用，比画面中的面孔或明亮区域更为关键。
Motion of an object inside the frame can draw attention so that a viewer is blinded to disruptions in the rest of the visual field. Levin & Varakin (2004) found that video clips can have disruptions in the form of a blank screen for as long as 0.6 sec without test subjects being aware of the interruption happening. The viewers in this study were so focused on the main narrative action that they were unable to recall that any disruption occurred.
Viewers who are not expecting any unusual continuity problems will not notice that an actor is replaced in the middle of a scene in a video clip, even when the two actors do not look alike and are wearing different clothes (Levin & Simons 1997). This effect can occur in real life as well. Experiments show that people in natural interactions with strangers may not notice when the person they are talking to is swapped with someone else during their interaction (Simons & Levin 1998).
These research results may explain why one aspect of continuity editing works. Editors can hide the visual discontinuity created by a cut by cutting on action, for instance, having the cut occurring in the middle of a motion of the actor that is holding the viewer’s attention (Dmytryk 1984, pp. 27-33). This also may explain why audiences seem to be so forgiving of continuity errors (Levin & Simons 2000).
这些研究结果可能从一个角度解释了连续性剪辑为什么能产生效果。剪辑师可以利用动作间的剪辑来掩盖剪切造成的视觉不连续性，比如在演员做一个吸引观众注意力的动作的中途进行剪辑 (Dmytryk 1984, pp. 27-33)。这也解释了为什么观众对连续性错误如此宽容 (Levin & Simons 2000)。
In our discussion of sequences from Samskara in Paper I, we highlight scene transitions that are masked by object motion. In the first example (Fig. 8, p. 36), the main object of focus is a pulsating trilobal pattern in the forward direction which attracts the viewer’s attention, and which, for many viewers, can hide the wipe that replaces the background.
Faces attract attention because we have evolved to be aware of socially relevant features such as human figures, many animal faces, and the “directed gaze” (where other people or animals are looking). This has been investigated for viewers of static artwork including paintings and sculptures (Buswell 1935; Yarbus 1967), as well as for dynamic imagery in film and video (Treuting 2006; Birmingham, Bischof, & Kingstone, 2008; Castelhano, Wieth, & Henderson, 2007; Võ et al. 2012, Smith 2013).
When a head is close enough to be seen in detail, the eyes draw more attention than other parts of the face (Treuting 2006). In fact, one of the rules for when to cut in continuity editing of shots with a person moving offscreen is to make the edit at the point immediately after the person’s eyes have left the frame: since a viewer tracks faces, any additional frames with just the person’s body in the frame, but without the face, is unnecessary (Dmytryk 1984, p. 30).
当观众能够清晰地看到角色头部时，角色的眼睛比脸部其他部位更能吸引观众的注意力 (Treuting 2006)。实际上，在连续性剪辑中，一个常用的剪辑技巧是在人物的眼睛刚离开画面的瞬间进行剪辑，因为观众习惯于追踪面孔，一旦面孔不在画面中，仅展示身体的画面就变得多余 (Dmytryk 1984, p. 30)。
Attention to the nose and mouth are also important, with the former used to track a moving face, and the latter when the person on-screen is speaking (Võ et al. 2012). Since faces are such a consistent draw, fulldome filmmakers can use them to not only draw attention, but to lead a viewer’s gaze in directions where they normally would not be looking.
鼻子和嘴巴也是重要的注意力焦点，鼻子有助于追踪移动的面孔，嘴巴则在银幕上的人物讲话时更加重要 (Võ et al. 2012)。由于面孔具有如此强大的吸引力，全景球幕电影的制作人可以在利用它们吸引注意力的同时，引导观众的视线转向他们通常不会注意到的方向。
Brightness 亮度
Visual signals traveling along nerve fibers away from the retina are sent to two different sets of cells that define the visual pathway: the magnocellular and parvocellular systems (Livingstone & Hubel 1988). One set of cells making up the brain’s magnocellular pathway is sensitive to changes in brightness and to high contrast, reacts quickly, but is otherwise colorblind and has low spatial resolution. The set of cells for the parvocellular pathway is used to process color information, is not sensitive to contrast, works slowly, and has high spatial resolution.
Experiments show that a change in brightness in the visual field will “pop out” immediately to an observer since such signals are processed quickly by the “magno” pathway, without the need for conscious focused attention. Changes in color, on the other hand, will not be noticed as quickly since they are sent through the slower “parvo” system (Theeuwes 1995). Treuting’s study (2006) using film clips confirm this since bright objects were more important than differences in color in attracting viewer attention.
Humans and other animals have defensive reactions to objects they appear to be on a collision course with (Schiff 1965). Human infants as young as 3-6 weeks will blink more and move their heads back in reaction to stimuli of a dark object that is approaching or “looming,” while no such effect occurs when the object is receding (Náñez 1968). In controlled experiments with adults, Franconeri & Simons (2003) showed that an object on-screen that was looming (but not receding), or in motion, or which appeared suddenly would receive greater attention. We would thus expect that an object that grows in size until it dominates on-screen to receive greater attention.
Treuting (2006) was able to show qualitative support for this hypothesis in an eyetracking experiment using a clip from the Quidditch match in Harry Potter and the Sorcerer’s Stone (2001), during which viewers shifted their gaze from a player in red who has the ball, to a player in green who is approaching from behind and filling the screen in the process.
If the lens focal length, lens aperture, and distance of the camera result in shallow focus, then only part of a scene will be in focus. The director can set the depth of field (DOF) to keep a foreground element in focus, with the background out of focus, or vice versa. The DOF increases with small apertures, while large apertures result in a shallower focus. Alternatively, switching to a narrow angle lens (e.g., telephoto lenses which have focal lengths 70 mm or higher) will result in less DOF, while a wide angle lens (35 mm or less) will result in more of the scene in focus.
A rack focus is a change in focus in the middle of a shot, resulting in a self-conscious change in emphasis, redirecting the audience’s attention from the foreground to the background or vice versa (Van Sijll 2005, pp. 14-15). Fulldome film footage shot live with a fisheye lens (which by definition is a very wide angle, short focal length lens) will have an extremely broad DOF. However, computer-generated animation based on 3D modeling is not limited to what is possible with traditional physical camera and lens setups (e.g., Cole et al. 2006), so rack focusing is possible for computer-rendered shots.
推拉焦（rack focus）是在单镜头内改变焦点，将观众的注意力从前景转移到背景，或从背景转移到前景 (Van Sijll 2005, pp. 14-15)。使用鱼眼镜头（广角、短焦镜头）拍摄的全景球幕电影具有极宽的景深。然而，基于 3D 建模的计算机生成动画不受传统摄影机和镜头设置的限制 (e.g., Cole et al. 2006)，因此计算机渲染镜头可以实现推拉焦效果。
Framing of space 空间构图
Cinematic staging, the way that directors arrange and move actors within the film frame, has been analyzed by Bordwell (2005). He outlines how directors throughout the history of cinema, shaped by the cinematic styles popular at the time and by available filmmaking technology, have employed different methods to direct the viewer’s gaze, including composition and staging within the frame, blocking of actors over the course of a shot or scene, lighting used to highlight people or objects in the scene, and movement of objects and actors to reveal information or emphasize focus.
Treuting (2006) tested how viewers responded to actual film clips using eye-tracking. One finding is that when a director visually withholds information, such as in Alfred Hitchcock’s Rope (1948), audience members will tend to seek to find it themselves by searching the frame. Rope is famous for having almost no visible cuts to suggest that it was shot in one continuous take. Instead of using edits to change the audience’s focus, the camera would instead pan to follow characters walking through the set; and instead of cutting to a close-up, the camera would move or zoom in closer to focus on a prop that the audience should be paying attention to.
Redirecting the camera towards one direction means that other characters and objects in the scene would be outside the frame. If visual information critical to the story is withheld, viewers will actively scan the frame in anticipation of seeing that important detail. Since immersive fulldome cinema is effectively frameless, cinematic staging is done without the edges of the frame to help constrain visual content. But as shown in the sample frame sequences below, there are multiple ways for directors to stage the camera and the scene to drive viewer attention.
A match cut refers to the bridging of two shots that have visual elements that match each other. More often they are used dramatically to link together different ideas, locations, or time periods represented by the individual shots (Van Sijjl 2005, pp. 118-131). The transition from prehistoric to spacefaring eras in 2001: A Space Odyssey is executed with one of the most famous match cuts in all of cinema: a shot of a bone tossed up into the air cuts to a satellite in orbit around the Earth. This cut manages to embody all of the different ways that match cuts can work.
画面匹配剪辑是一种通过视觉元素的相似性将两个镜头巧妙连接起来的技巧，这种技巧常用于戏剧性地将不同的观点、地点或时间段联系起来 (Van Sijjl 2005, pp. 118-131)。比如在《2001: 太空漫游》（2001: A Space Odyssey，1968）中，从将骨头抛向空中的镜头剪切到绕地轨道运行的卫星镜头，就是影史最著名的匹配剪辑之一，它巧妙地展现了匹配剪辑的多种可能性。
In addition to matching the physical shape and motion of the two objects, the cut represents a leap to a different time and place, as well as conveying the evolution of weaponry, from a bone used as a club to nuclear weapons in orbit (a concept dropped from the final film; Walker, Taylor, & Ruchti 1999; pp. 181-182).
Editing that follows the rules of Hollywood continuity also has a tendency to lead to “edit blindness” where the audience does not notice a cut has occurred. Smith & Henderson (2008) have found that cutting together two shots that have matching action—such as a character reaching for a door and a cut to a close-up of a hand grasping the doorknob—are more “invisible” to the viewer.
Cuts matching action are found to be the most seamless if there is a slight repeat of the action for several frames from the first cut in the second cut (Shimamura, Cohn-Sheehy, & Shimamura 2014). Viewers are able to re-orient their attention after a cut if the visual content between the two shots are similar enough (Valuch et al. 2014). For fulldome films, a match cut can be used to maintain viewer focus in the same direction in the dome after the cut as before the cut.
为了实现更自然的过渡，如果第二个剪辑开始时能够轻微重复第一个剪辑中的几格动作，这样的动作匹配剪辑会显得更加流畅 (Shimamura, Cohn-Sheehy, & Shimamura 2014)。此外，如果两个镜头之间的视觉内容足够相似，观众就能够在剪切后迅速调整注意力 (Valuch et al. 2014)。在全景球幕电影中，可以通过匹配剪辑使观众在剪辑前后的注意力保持在同一方向。
Color variation 色彩变化
An object or part of a scene that changes color can draw attention, although this is subordinate to other cues such as motion (Smith 2013). Similarly, Buswell (1935) found that when viewing static pictures, colors that were pronounced drew less than expected amount of attention, and Tatler, Baddeley, & Gilchrist (2005) confirmed this in a modern eye tracking experiment with photographs.
Franconeri & Simons (2003) found that color was one factor that did not lead to greater attention from the adult test subjects in their experiments. The cone photoreceptor cells most sensitive to color are clustered near the center of the fovea of the retina with sharp dropoffs in cell density just 10° away from the center (Purves et al. 2001). Therefore, changes in color will not be as detectable in a viewer’s peripheral vision.
Although color is less powerful as a way to draw attention, there are films that are predominantly black and white, but shows objects on-screen in full color (e.g., the flag in Battleship Potemkin, the girl with the red coat in Schindler’s List, the splashes of color throughout Sin City). In extreme examples like these, the contrast of an object saturated with color against a monochromatic background can draw immediate attention if the viewer is looking in the same direction.
For straight-ahead motion, we theorize that the point in the visual field where the camera is headed (or the “focus of expansion”) will draw attention. Similarly when the camera is moving forward while turning, there will be a tendency for the viewer to focus in the direction of the turn.
我们推测，在摄影机直线向前运动时，观众视线会被吸引到摄影机前进方向的焦点上，即视觉场中的“扩张/扩展焦点”（“focus of expansion, FoE”）。类似地，当摄影机在向前移动的过程中转向时，观众的注意力也会自然地跟随转向。
This is supported by eye-tracking experiments that followed drivers navigating vehicles on actual streets. For curved roads, drivers directed their gaze into the turn, instead of merely relying on their peripheral vision (Shinar, McDowell, & Rockwell 1977; Land & Lee 1994).
Treuting (2006) showed that viewers watching a movie clip with a point-of-view shot of a foot chase through a city tended to focus ahead. However, since the narrative involved the main character hunting a criminal suspect, viewers would often redirect their focus to people and objects that were passing by to see if they matched the target of the chase.
In the case of many astronomical fulldome films, when a camera moves forward through a three-dimensional star field, the stars translate forward towards the viewer, with the translation vectors pointing back to the focus of expansion in front of the camera. Although not experimentally verified, we hypothesize that this vanishing point will naturally draw a viewer’s eyes if there is otherwise nothing visually distinctive about the optic flow of stars streaming past. If the camera is turning while moving forward, we speculate that the audience gaze will re-focus towards the turn.
In this section, we examine five different sequences by examining original domemaster frames from three fulldome films: Black Holes: The Other Side of Infinity (2006) and Dynamic Earth (2012), both directed by Tom Lucas, as well as Habitat Earth (2015), directed by one of the authors (Wyatt). By looking at these sequences, which include those that consist of multiple shots edited together as well as shots that consist of continuous motion, we will see how the audience’s gaze can be drawn by elements within the shot, through framing devices, and with match cuts.
在本节中，我们深入分析了三部全景电影：《黑洞：无限的另一面》（Black Holes: The Other Side of Infinity，2006），《动态地球》（Dynamic Earth，2012），以及《地球：万物的栖息地》（Habitat Earth，2015），从中选择了五组不同的镜头片段。前两部作品由 Tom Lucas 执导，而第三部则由本文作者之一 Wyatt 执导。这些镜头片段包括了由多个镜头组成的复杂场景以及单一镜头中的连续运动，通过仔细审视它们，我们可以了解到观众的注意力是如何被镜头中的视觉元素、构图技巧和匹配剪辑所引导的。这种分析有助于揭示全景球幕电影中视觉叙事的构建方式，以及如何有效地抓住并保持观众的兴趣。
The first sequence is the simplest of our examples: live film footage of the Delta II rocket taking the Swift Gamma Ray Burst Mission satellite into orbit from Cape Canaveral (Fig. 11). The rocket is lit up at night and the rest of the domemaster frame is in complete darkness [2]. Because most of the domemaster is black, the eye is drawn to the brightly light rocket and gantry in the forward center part of the frame. The motion of the rocket and the slow brightening of the lights also help to draw the audience’s attention.
本片的第一组镜头片段是以下例子中最简单的，展现了 Delta II 火箭搭载 Swift 伽马射线暴（Swift Gamma Ray Burst Mission）任务卫星进入卡纳维拉尔角上空轨道的实景拍摄镜头（见图 11）。夜空中的火箭被照亮，而画面的其余部分则沉浸在一片漆黑之中。由于球幕母版画面大部分是黑色的，因此观众的视线自然会被吸引到画面前方中央明亮的火箭和发射架上。火箭的运动轨迹和灯光从暗到亮的变化也有助于吸引并保持观众的注意力。
The subsequent launch footage of the Delta II rocket intercuts reaction shots of scientists involved with the Swift mission on the ground as they see the rocket go up, with telephoto footage of the rocket rising up from the launch pad and in the air. The first shot is from the ground with the camera tracking from right to left, and pointed back at a row of scientists standing on the ground in Florida (Fig. 12a). Many of the faces are hidden by binoculars, so we are drawn to the faces that we can see, including one man in the white t-shirt who benefits from the viewers’ center bias by standing just slightly to the left of center.
接下来的发射画面中，我们看到了与 Swift 任务相关的科学家们在地面上的反应，他们目睹了火箭升空的瞬间，以及火箭从发射台腾空而起的长焦镜头。首个镜头捕捉了佛罗里达州地面上一排科学家的侧影，他们中的许多人正通过望远镜观看发射（图 12a）。由于大部分面孔被望远镜遮挡，观众的注意力自然会被那些未被遮挡的面孔所吸引，尤其是那位穿着白色 T 恤、站在中心略偏左位置的男士，他恰好处于观众视觉偏好的中心区域。
We cut to the rocket taking off after the countdown clock reaches zero. The rocket is in the front center part of the frame, matching the location of the previous subject with a visible face before the cut (Fig. 12b).
In Fig. 12c, we cut back to the crowd to see their ecstatic reaction shots. The camera tracks back in the opposite direction, and again most of the faces are hidden behind binoculars and cameras. The single visible face belongs to the man in glasses, white hat, and yellow shirt who has a wide grin across his face. His face and position in the front center part of the frame make him the focus of attention.
In Fig. 12d, we cut again but the camera stays on the ground. We are on the other side of the crowd, seeing the rocket plume where the smiling man was in the previous shot. The cut has matched the position of where we were looking before with the rocket plume that we need to focus on in the far distance.
To help direct the viewer’s attention, there are individuals with their backs turned who are on either side of the rising rocket. Since we do not see any faces, the people in the shot act as framing architecture to direct our attention. Everyone in the shot is also looking at the rocket via binoculars or with their normal unaided vision. Since we have a tendency to look where people are gazing, we follow the gazes of the crowd at Cape Canaveral as well.
This is a sequence that we covered in Yu et al. (2007), but we repeat it here in more detail. The sequence consists of a rendered flight through a simulation of the Milky Way Galaxy. During the course of the flight, a star turns into a red giant, and then explodes into a supernova. Given the vast number of stars visible in the domemaster, is it possible to get the audience to notice the red giant before it explodes?
The start of the sequence (Fig. 13a) has the camera flying through the Milky Way, banking and turning to the right. We have a natural propensity to turn our gaze to the right as well, trying to anticipate what is coming “just around the corner”—or in this case, just beyond the right edge of the domemaster.
The bank and turn ends and the camera levels out and flies forward. Our gaze also levels out and re-centers towards the direction the camera is headed towards. The stars whizzing past point back to this vanishing point in the distance. Normally the band of the Milky Way and the Magellanic Clouds would attract attention by being bright objects against a dark backdrop. However the latter are in the upper left of the domemaster, meaning they are far in the back and hence not noticeable to an audience facing forward. The Milky Way cuts across the domemaster nearly at the halfway mark, meaning it is high up, again making it hard to see. Both the composition and the camera motion is leading the viewer’s eyes to a point below the Galactic Center. And as the camera continues to move forward, a star to the right of this direction turns into a red giant (Fig. 13b). The dynamic color shift and change in size is very noticeable when all of the other stars remain monochromatic points. The red giant expands, and then suddenly explodes into a supernova in a bright flash (Fig. 13c), followed by the expanding supernova remnant. The change in brightness, size, and varied colors of the expanding supernova remnant all serve to highlight this object (Fig. 13d) so that if you miss it earlier in the sequence, multiple dynamic cues will eventually draw your eyes to the exploding star.
当倾斜和转弯结束，摄影机平稳飞行，我们的视线也会随之平稳，并重新集中在摄影机前进的方向。飞速掠过的恒星似乎都指向远处的消失点。通常，银河系的明亮带（the band of the Milky Way）和麦哲伦星云（Magellanic Clouds）作为黑暗背景下的明亮主体会吸引观众的目光，但它们位于球幕母版的左上方，对于面向前方的观众来说太远了，不太容易注意到。银河系在球幕母版上的位置接近中点，因此也处于较高的位置，不易观察。构图和摄影机运动将观众的视线引向银心（Galactic Center）下方的某一点。随着摄影机的前进，这个方向右侧的一颗恒星转变成了红巨星（图 13b）。在所有其他恒星都保持为单色点的情况下，这颗恒星颜色的动态变化和体积的增大非常引人注目。红巨星随后膨胀，并在一次突然的明亮闪光中爆炸成超新星（图 13c），接着是超新星遗迹的膨胀。超新星遗迹的亮度变化、体积增大、颜色变化，都使其成为焦点（图 13d），即使观众在这段镜头早期错过了这一变化，多个动态线索最终也会将他们的目光吸引到这颗爆炸的恒星上。
In a sequence showing undersea currents starts above Earth with the currents visualized as moving arrows that twist and follow surface and underwater barriers, circulate, diverge, and converge. Currents near the surface are colored white while deeper ones are colder and hence colored blue and purple. The camera is above North America long enough for the audience to recognize familiar features like the Florida peninsula before plunging into the water below (Fig. 14a).
Once underwater, a careful viewer will discern the continental shelves and underwater seamounts that can deflect currents. However what is most noticeable are the bright white arrows swirling just beneath the surface. But because the camera lies below these currents, our gaze is turned to the eddies of currents above us, drawn by both the bright color as well as the frenetic activity of the arrows (Fig. 14b).
For the unsuspecting viewer still mesmerized by the white surface arrows, there will be a surprise as a school of blue to purplish arrows are headed straight for the audience from the far right, looming larger in size as they approach (Fig. 14c). The viewer may not notice this until the arrows “collide” with and slip around the virtual camera. At this point they are impossible to ignore because they appear as if they are aiming for the viewer, and are more dominating (larger and faster) than the white surface arrows which now become lost behind the flurry of foreground activity (Fig. 14d).
In planetarium visualization software, it is commonplace to zoom out from the planets in the Solar System to stars at the interstellar scale, and finally clusters of galaxies at the intergalactic scale. Objects at the next scale are either visible or have their visibility fade up as the camera zoom approaches the scale associated with that object (e.g., galaxies from the Sloan Digital Sky Survey not appearing until the camera is millions of light years from Earth).
在天文馆可视化软件中，经常采用变焦手法，从太阳系行星缩放到星际尺度的恒星，再缩放到星系尺度的星系团。随着镜头变焦到下一尺度，该尺度下的物体会变得可见，或逐渐提高可见度（例如斯隆数字巡天（Sloan Digital Sky Survey）能拍到的星系，直到镜头远离地球数百光年时才会显现）。
In this last sequence from Dynamic Earth, a zoom through multiple underwater scales to view different animals found at those scales uses a new visual paradigm of a rack focus instead of physical movement to denote the change of scale. Since the smallest scale is at the level of photosynthetic dinoflagellates (at tens of microns across) that forms the base of the food chain in the oceans (Fig. 15a), such a transition would appear to mimic the re-focusing of microbial life captured between slides in a microscope. Before the re-focusing, the next scale is evident as blurred forms in the background behind the foreground creatures.
After the camera re-focuses/zooms to the next scale, copepods, tiny crustaceans 1-2 mm in length, become clear (Fig. 15b). Like the dinoflagellates before them, the copepods are immersed within ocean water. Instead of merely floating and drifting, the copepods propel themselves forward, including one that hits the virtual camera “lens” with a noticeable thump in the soundtrack (Fig. 15c).
The next change in scale shows octopus larvae, less than 1 cm in length (Fig. 15d). As with all of the fauna shown so far in this sequence, the individuals rendered are floating or self-propelling through the water in every direction throughout the domemaster. The audience’s gaze is not being redirected to any one location in the dome. All directions are equally likely to have activity.
The one exception is the copepod that bumps up against the camera lens, which is doing so against the forward center part of the domemaster, making its activity easy to spot by the viewer.
唯一的例外是一只桡足类触碰到了摄影机镜头，这一动作发生在球幕的前方中心部分，观众很容易注意到。
The next jump in scale reveals a school of black jack, a large ocean fish that can measure up to a meter in length. They swim in front of and around the camera. The eye is easily drawn to nearby individuals who dominate the scene when they swim close enough to be illuminated by (presumably) a lamp associated with the virtual camera (Fig. 15e).
However when the school of fish is more distant, the grouping of darkened fish (no longer lit by the virtual light source) swimming in unison draws attention. The disk of the sun, distorted in appearance when viewed from beneath the surface of the waves, is also prominent since it is the only other light source in the scene.
As the school of jacks disappear, they are replaced by one, followed by many, hammerhead sharks (Fig. 15f). We have now reached a size scale of several meters. Although they are fewer in number, the sharks still fill up the dome impressively (Fig. 15g).
The end of the sequence nears as the hammerhead sharks disappear except for a handful that swim near the top of the dome (corresponding to the center of the domemaster). Since these are just at the edge of the audience’s visual field (even for those sit at the back of the room), they help frame and highlight the large space in the forward part of the dome where no sharks are swimming (Fig. 15h). Even the light from the sun dims as the camera plunges deeper into the ocean.
The large negative space becomes a compelling area of focus by dominating the frame and being in the front center. The audience is finally rewarded with the beak of a large baleen whale that emerges from the murk, slight right of forward center. As the whale swims gracefully through the scene, the sun has diminished into a faint smudge. Our attention is captured by the whale, lit by the only bright light left, the virtual spotlight associated with the camera. The illumination catches the large eye, which, being eye-like, also becomes the focus of the viewer (Fig. 15i).
In the California Academy of Sciences production Habitat Earth, the show introduces a kelp forest early in the show, building a set of concepts that will be integral to ideas developed later in the story. The kelp forest is introduced with the camera moving forward through an underwater depiction of San Francisco Bay, as a wipe transition reveals the clearer, bluer waters with kelp visible in the distance (Fig. 16a).
As the camera enters the kelp forest, its forward motion slows, inviting viewers to look around and feel immersed in the environment; the narration describes the kelp forest in general terms, without drawing attention to any particular elements in the scene.
A slight camera tilt downward accompanies a reference to kelp fronds on the seafloor (a significant source of nutrients), but only when a school of fish moves rapidly from the left into the mid-ground (Fig. 16b) is the viewers’ attention pulled toward a specific location on-screen, timed to a reference to “spiny purple sea urchins” in the narration. The combination of movement and contrasting color ensure that the audience is looking at the right place at the right time.
The reappearance of the school of fish (this time moving from right to left in the frame) draws attention toward the center of the frame just in time for a sea otter to appear in the distance (Fig. 16c). The sea otter’s movement and high contrast facial features maintain its focus as the center of interest (Fig. 16d), and the camera naturally follows the otter as it ascends to the water’s surface, where it joins a second sea otter to consume a sea urchin snack (Fig. 16e).
Although the change in scenery (from below water to above water) and relatively motionless camera invite the audience to look around and enjoy the environment in a relatively undirected fashion, the pull of the mammals’ faces and behavior keeps attention focused on the sea otters, as the narration describes their importance to the proper functioning of the kelp forest ecosystem.
When the camera follows the second sea otter back underwater, the creature’s path crosses the center of interest in the front of the theater, and in its perceptual wake, graphic elements appear to illustrate various concepts in the narration (Fig. 16f). The saturated green, yellow, and orange of these elements contrasts with the “natural” hues in the scene, effectively directing the viewers’ attention to the relevant content on-screen.
The sequence continues with fluid camera motion highlighting various vignettes in the kelp forest, maintaining the focus in the center of interest (Figs. 17a, 17b). In order to illustrate a rapid change in scale from centimeters to microns, the virtual camera racks focus in a style similar to the Dynamic Earth undersea fauna sequence described above (Fig. 17c). After returning to a macroscopic scale, we reprise the highly saturated graphic elements that appeared earlier in the scene, illustrating the connections between different species in a complex food web (Fig. 17d). Minimal camera motion once again invites the audience to examine the entire scene, underscoring the concept that these environments are highly complex. As the kelp forest sequence draws to a close, the narration makes a critical point about the Sun being the energy source for the biodiversity we have witnessed in the scene, and a lack of large objects in the center of interest coupled with the brightness of sunlight refracting through the water’s surface draws viewers’ attention to the location of the Sun on the dome (Fig. 17e). This precedes a crossfade to the next scene in the show, to a terrestrial forest (Fig. 17f). The similar positioning of the Sun at the end of the kelp forest sequence and the beginning of the terrestrial forest sequence reinforces continuity between the two scenes and also emphasizes the critical content point that the Sun provides energy for virtually all living systems on Earth.
Despite having the substantial freedom to direct their gaze anywhere in the dome, audiences often do not. Part of this reason may be due to center bias, where we naturally return our gaze to what is directly in front of us. But clever directors and producers can also redirect audience attention. The most important visual cues they can use are motion followed by human (or animal) faces; less important are secondary cues like brightness and color variations, and “looming” objects.
It may be possible for one moving object to direct focus so effectively that the rest of the visual environment is ignored. When changes occurring elsewhere in the visual field are finally noticed, they can be a surprise to the viewer. The power of motion and other visual cues to draw attention may explain why faster paced giant screen documentaries and Hollywood narrative films shown in IMAX theaters (Paper I, pp. 31-34) are still comprehensible to audiences.
These types of cues become more important for productions not intended for tilted fulldome theaters. In unidirectional theaters, the bias to the center defines an easy “default” direction for the audience to look. In level theaters with concentric seating, the “default” view may shift to the zenith of the dome, but depending on the seat tilt, there may not be a common point in the dome that is easily visible to all of the audience.
However, even in this latter case, the visual cueing discussed in this paper can be used to draw viewer attention. Merely “panning” the object of interest around the theater can draw audience attention because of its motion. In virtual reality (VR), the full visual field is spherical instead of hemispherical. Yet almost all consumer head-mounted displays have limited fields-of-view that are roughly 100° across, far smaller than fulldome (Shanklin 2016). Therefore when developing content that can also play in VR as well as fulldome, it is even more important for visual cues be used to effectively guide the viewer.
The cues we have discussed in this paper are suggestive based on anecdotal evidence, earlier findings in human perception, and work by cognitive psychologists studying how viewers react to traditional cinema. We do not have comparable research to show how effective such cues are in actual fulldome films shown in real-life settings. However it appears that large-scale eye-tracking experiments in a dome theater are possible (Bielecki, Potęga vel Żabik, & Gochna 2016). New empirical studies based on their approach will help shore up our results, and place any recommended best practices for immersive cinema on a solid research footing.
本文讨论的视觉线索是以轶事证据（anecdotal evidence），对人类感知的早期发现，以及认知心理学家对传统电影观众反应的研究为基础的。目前尚未有比较研究来证明这些线索在实际全景球幕影院放映的效果如何。不过，在球幕影院进行大规模的眼动追踪实验应该是可行的 (Bielecki, Potęga vel Żabik, & Gochna 2016)。基于这一方法展开的新的实证研究将有助于支撑我们的结果，并将沉浸式电影的任何推荐最佳实践建立在坚实的研究基础之上。
Acknowledgments 致谢
We thank Tom Lucas, Donna Cox, and Mike Bruno for permission to use images from their films. We also thank Kathy Honda for help with our library research.
感谢 Tom Lucas，Donna Cox 和 Mike Bruno 允许我们使用他们电影中的静帧。感谢 Kathy Honda 帮助我们进行图书馆文献搜索。
References 参考文献
Bielecki, M., Potęga vel Żabik, K., & Gochna, M. 2016, “Mass Measurement of Eye Movements Under the Dome–Proof of Concept Study,” paper presented at the meeting of the International Planetarium Society, Warsaw, Poland.
Birmingham, E., Bischof, W. F., & Kingstone, A. 2008, “Gaze selection in complex social scenes,” Visual Cognition, 16, 341–355.
Bordwell, D. 2005, Figures Traced in Light: On Cinematic Staging, Berkeley, California: University of California Press.
Bordwell, D. 2006, The Way Hollywood Tells It: Story and Style in Modern Movies, Berkeley, California: University of California Press.
Bordwell, D. & Thompson, K. 2006, Film Art: An Introduction, New York: McGraw-Hill.
Bradbury, A. 2016, “Domography: Leading the eye around the dome,” paper presented at the meeting of the International Planetarium Society, Warsaw, Poland.
Brasel, S. A., & Gips, J. 2008, “Points of view: Where do we look when we watch TV?” Perception, 37(12), 1890–1894.
Buswell, G. T. 1935, How People Look at Pictures: A Study of the Psychology and Perception in Art, Chicago, Illinois: University of Chicago Press.
Castelhano, M. S., Wieth, M. S., & Henderson, J. M. 2007, “I see what you see: Eye movements in real-world scenes are affected by perceived direction of gaze,” in L.P.A.E. Rome (Ed.), Attention in Cognitive Systems, Berlin: Springer, pp. 252–262.
Cole, F., DeCarlo, D., Finkelstein, A., Kin, K., Morley, R. K., & Santella, A. 2006, “Directing Gaze in 3D Models with Stylized Focus,” in T. Akenine-Möller and W. Heidrich (Eds.), EGSR ‘06: Proceedings of the 17th Eurographics Symposium on Rendering Techniques, Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, pp. 377–387.
Dmytryk, E. 1984, On Film Editing, Boston: Focal Press.
Dorr, M., Martinetz, T., Gegenfurtner, K. R., & Barth, E. 2010, “Variability of eye movements when viewing dynamic natural scenes,” Journal of Vision, 10(10): 28, 1–17.
Franconeri, S. L., & Simons, D. J. 2003, “Moving and looming stimuli capture attention,” Perception & Psychophysics, 65(7), 999–1010.
Franconeri, S. L., Hollingworth, A., & Simons, D. J. 2005, “Do new objects capture attention?” Psychological Science, 16(4), 275–281.
Goldstein, R. B., Woods, R. L., & Peli, E. 2007, “Where people look when watching movies: Do all viewers look at the same place?” Computers in Biology and Medicine, 37(7), 957–964.
Land, M. F., & Lee, D. N. 1994, “Where do we look when we steer,” Nature, 369, 742–744.
Le Meur, O., Le Callet, P., & Barba, D. 2007, “Predicting visual fixations on video based on low-level visual features,” Vision Research, 47(19), 2483–2498.
Levin, D. T., & Simons, D. J. 1997, “Failure to detect changes to attended objects in motion pictures,” Psychonomic Bulletin & Review, 4(4), 501–506.
Levin, D. T., & Simons, D. J. 2000, “Perceiving stability in a changing world: combining shots and integrating views in motion pictures and the real world,” Media Psychology, 2(4), 357–380.
Levin, D. T., & Varakin, D. A. 2004, “No pause for a brief disruption: Failures of visual awareness during ongoing events,” Consciousness and Cognition, 13(2), 363–372.
Livingstone, M., & Hubel, D. 1988, “Segregation of form, color, movement, and depth: Anatomy, physiology, and perception,” Science, 240(4853), 740–749.
Mital, P. K., Smith, T. J., Hill, R. L., & Henderson, J. M. 2011, “Clustering of gaze during dynamic scene viewing is predicted by motion,” Cognitive Computation, 3(1), 5–24.
Náñez, J. 1988, “Perception of impending collision in 3-to 6-week-old human infants, Infant Behavior and Development, 11(4), 447–463.
Pausch, R., Snoddy, J., Taylor, R., Watson, S., & Haseltine, E. 1996, August, “Disney’s Aladdin: First steps toward storytelling in virtual reality,” in Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, New York: ACM, pp. 193–203.
Rodigast, R., & Gaston, L. M. 2010, “The Sound of Space: A Look at the History and Future of Audio in Planetariums.” In International Planetarium Society Conference Proceedings, Vol. 26, 73–77.
Schiff, W. 1965, “Perception of impending collision: A study of visually directed avoidant behavior,” Psychological Monographs: General and Applied, 79(11), 1–26.
Schwan, S., & Ildirar, S. 2010, “Watching Film for the First Time: How Adult Viewers Interpret Perceptual Discontinuities in Film,” Psychological Science, 21(7), 970–976.
Shimamura, A. P., Cohn-Sheehy, B. I., & Shimamura, T. A. 2014, “Perceiving movement across film edits: A psychocinematic analysis,” Psychology of Aesthetics, Creativity, and the Arts, 8(1), 77–80.
Shinar, D., McDowell, E. D., & Rockwell, T. H. 1977, “Eye movements in curve negotiation,” Human Factors, 19(1), 63–71.
Simons, D. J., & Levin, D. T. 1998, “Failure to detect changes to people during a real-world interaction,” Psychonomic Bulletin & Review, 5(4), 644–649.
Smith, T. J. 2013, “Watching you watch movies: Using eye tracking to inform film theory,” in A.P. Shimamura, Psychocinematics: Exploring Cognition at the Movies, New York: Oxford University Press, pp. 165–191.
Smith, T. J., & Henderson, J. M. 2008, “Edit Blindness: The relationship between attention and global change blindness in dynamic scenes,” Journal of Eye Movement Research, 2(2), 1–17.
Stelmach, L. B., Tam, W. J., & Hearty, P. J. 1991, June, “Static and dynamic spatial resolution in image coding: An investigation of eye movements,” SPIE Vol. 1453: Human Vision, Visual Processing, and Digital Display II, 147–152.
Tatler, B. W. 2007, “The central fixation bias in scene viewing: Selecting an optimal viewing position independently of motor biases and image feature distributions,” Journal of Vision, 7(14), 1–17.
Tatler, B. W., Baddeley, R. J., & Gilchrist, I. D. 2005, “Visual correlates of fixation selection: effects of scale and time,” Vision Research, 45(5), 643–659.
Theeuwes, J. 1995, “Abrupt luminance change pops out; abrupt color change does not,” Perception & Psychophysics, 57(5), 637–644.
Tosi, V., Mecacci, L., & Pasquali, E. 1997, “Scanning eye movements made when viewing film: preliminary observations,” International Journal of Neuroscience, 92(1–2), 47–52.
Treuting, J. 2006, “Eye tracking and the cinema: A study of film theory and visual perception,” SMPTE Motion Imaging Journal, 115(1), 31–40.
Tseng, P. H., Carmi, R., Cameron, I. G., Munoz, D. P., & Itti, L. 2009, “Quantifying center bias of observers in free viewing of dynamic natural scenes, Journal of Vision, 9(7), 1–16.
Valuch, C., Ansorge, U., Buchinger, S., Patrone, A. R., & Scherzer, O. 2014, June, “The effect of cinematic cuts on human attention,” in TVX ‘14: Proceedings of the 2014 ACM International Conference on Interactive Experiences for Television and Online Video, New York: ACM, pp. 119–122.
Van Sijll, J. 2005, Cinematic Storytelling: The 100 Most Powerful Film Conventions Every Filmmaker Must Know, Studio City, California: Michael Wiese Productions.
Võ, M. L. H., Smith, T. J., Mital, P. K., & Henderson, J. M. 2012, “Do the eyes really have it? Dynamic allocation of attention when viewing moving faces,” Journal of Vision, 12(13), 1–14.
Walker, A., Taylor, S., & Ruchti, U. 1999, Stanley Kubrick, Director: A Visual Analysis, New York: W. W. Norton.
Wollen, T. 1993, “The bigger the better: From CinemaScope to Imax,” in P. Hayward and T. Wollen (Eds.), Future Visions: New Technologies of the Screen, London: BFI Publishing, pp. 10–30.
Yarbus, A. L. 1967, Eye Movements and Vision, New York: Plenum Press.
Yu, K.C., Brownell, M., Schoemer, J., Neafus, D., Lucas, T. & Zager, Z. 2007, “Live Action Film Footage for an Astronomy Fulldome Show,” Planetarian, 36(1), 6–17, 86–87.
Yu, K.C., Neafus, D., & Wyatt, R. 2016, “Filmmaking for the Fulldome: Best Practices and Guidelines for Immersive Cinema, Part I,” Planetarian, 45(4), 26–32, 34, 36, 38–39.
1.This could be due to an intrinsic desire to focus in the forward direction, or as Pausch et al. noted, perhaps the participant in the ride was unfamiliar enough with the technology not to know to look to the side. This unfamiliarity correlates with a potential technology generation gap, where even today, we see anecdotal evidence that older users who try out consumer virtual reality headsets do not move around and engage as much as younger users (Willer 2016). 这种趋势可能是因为人们天生更倾向于注视前方，或者如 Pausch 等人所指出的，可能是因为参与者对技术不够熟悉，不知道应该环顾四周。这种对技术的不熟悉可能与潜在的技术接受度差异有关：即便在今天，我们也观察到老年用户在使用消费级虚拟现实头显时，他们的活动和参与度通常不如年轻用户那样高 (Willer 2016)。 ↩
2.In fact, the footage was originally shot in HD format, not fisheye. It was composited into the fisheye with black painted over imagery missing from the real shot. 实际上，这组镜头本来是用高清格式拍摄的，没有使用鱼眼镜头。 ↩
出处：Planetarian, 2016, 45(4) 标题：Filmmaking for Fulldome: Best Practices and Guidelines for Immersive Cinema (Part I) 作者：Ka Chun Yu, Dan Neafus, Ryan Wyatt 翻译：Horace Lu
（注：键盘快捷键“w”或左侧菜单右上角按钮，可切换文章列表视图与大纲视图）
[TOC]
Introduction 简介
Over a century of cinematic history has produced not only an enormous catalogue of films, but also a rich understanding of how this medium has embedded itself into our cultural landscape. The historical, sociological, and economic reasons for the rise of moviegoing are well documented (e.g., Schatz 1996; Thompson & Bordwell 2009). However, narrative filmmaking also has become as successful as it is today because filmmakers discovered ways to engage viewers with visual storytelling. How we perceive and interpret not only simple visual elements - such as light, shape, and color, but also the more complex interplay of these inside the two-dimensional frame and within three-dimensional spaces—help shape the aesthetics of cinema, and, in turn, communicate information and emotions (Zettl 1998).
Master filmmakers understand how to combine and link such elements into a visual structure that supports a story throughout a film (Block 2008). Audiences are drawn to cinematic entertainment (as well as its artistic descendants such as television), in part, because of the effectiveness of the visuals in adding to the experience.
Fulldome cinema is another offspring of traditional cinema, with antecedents in the OMNIMAX® (now IMAX Dome®) films of the 1970 s and 1980s, but whose flowering as a medium only began after the turn of this century. Although rooted in flat screen film, the fulldome medium has unique attributes that suggest different rules for effective storytelling. Informal guidelines have been adopted by fulldome filmmakers, but so far, to the best of our knowledge, these have not been codified.
In this first of a two-part paper, we make an initial attempt at doing so by reviewing past precepts and identifying our own prescriptions. We describe the critical ways in which the fulldome medium is different from traditional film and what that means for the filmmaker. We briefly illustrate the history of pacing in feature Hollywood films, giant screen cinema, and fulldome films; how they have evolved over time; and their consequences for filmmaking. If we accept that fulldome films are a type of immersive cinema in which the audience feels they are experiencing what is shown on screen, then it is possible to create a new theoretical framework of cinematic language and techniques which parallel those in traditional framed film. We outline the key elements of such a system of thinking, and show some examples from films that accommodate this theory. Finally, a dome display gives more directions where visual content can show up; viewers can miss critical on-screen information if they gaze in the wrong direction at the wrong time.
In Part II of this paper, we will identify directorial choices about what and how content is shown in order to mitigate this problem.
在论文的第二部分中，我们将探讨导演在展示内容时如何进行艺术抉择，以减少上述问题。
In both parts of this paper, we use actual sequences from fulldome films as examples for our analyses, and show frames from them to illustrate our points. Although there are many shows and producers that could have been highlighted, we chose films that we could carefully view while researching this paper, and hence represent a small but (we hope) representative fraction of all produced films.
Note that we are not attempting to describe how a filmmaker should work. We make no recommendations about what camera equipment or any other hardware to use, nor do we describe any modeling and animation software, nor any detailed production pipelines. The two papers in this series will describe a few useful tools that will impact how a production proceeds, but their primary goal is to get directors and producers to consider why visuals work the way they do in fulldome, and to think about structuring their shotswhether live action or computer-generated to create the greatest impact for the story they wish to tell.
Figure 1: Profile views of a (a) level fulldome theater, (b) a tilted fulldome theater, and (c) a giant screen theater. Unless specified otherwise, all illustrations by authors.
Fulldome cinema in domed theaters is a very different medium than traditional cinema: instead of projections on a flat screen in front of the audience, the hemispherical display surrounds the viewers, immersing them in imagery and sounds. Depending on whether the theater is level like a traditional planetarium (Fig. 1a), or is tilted with seats placed at an incline (Fig. 1b), audience members have their visual fields filled by the dome. In fact, unlike a traditional theater, imagery can extend to 180° in both the horizontal and vertical directions, far beyond what the viewer can take in at once.
By comparison, an IMAX® screen (Fig. 1c) fills a lateral field-of-view (FOV) of 60° to 120°, and a 40° to 80° vertical field, while the hemispherical OMNIMAX® expands on this to 180° laterally but only 130° vertically (Shaw & Douglas 1983).
Having such an expansive screen for projected imagery means that many more pixels are required to fill it. The highest resolution domes today can be filled with a fisheye reference domemaster frame that stretches at least 8,000 pixels across multiple meridians (e.g., the “True 8K” projection system at the Houston Museum of Natural Science’s Burke Baker Planetarium; n.a. 2016.).
Figure 2: Comparing the resolutions of an 8K domemaster with 4K cinema projection, high-definition TV (HDTV), and standard definition (SD) video. For a dome tilted by 25°, a viewer in the center of the dome looking forward would focus on the dot 25° above the front bottom (spring line) of the dome. Overlaid is the viewer’s visual field as indicated by the green (right eye) and orange (left eye) zones, where the overlap between the two is where binocular vision occurs. The lower boundary of the visual field is 40° below the front edge of the domemaster. The diameter of the dot roughly represents the size of foveal vision, where visual acuity is the greatest.
In today’s digital cinemas, 4K projectors have resolutions of 4096x2160 (Jukic 2016). If we compare the resolution that is visible to an audience member in a dome theater, who cannot see the entire domemaster at once, but can only have a binocular view (with both eyes) of 180°x110° at a time (Dragoi 1997), then the visual field visible in the 8K domemaster in the forward direction has greater resolution than that of a 4K cinematic frame (Fig. 2). But even with such a wide visual field, the viewer only sees content at high visual acuity (or resolution) in the foveal part of the retina, which is only a few degrees across (represented by the black dot in Fig. 2). Our perception of the world around us is built up piecemeal by the dynamic rapid scanning and fixation with the fovea in the eyes (Yarbus 1967).
The domes of fulldome theaters range from a few meters to 30 meters in diameter. Even the smallest theaters are highly immersive, since the dome’s display surface can fill the FOV immediately visible to a viewer. The sheer physical size of the domed theater (and the similar scale of the displayed imagery) is one of the most memorable elements as noted by audience members in surveys. For instance, 22% of 161 written audience responses from the general public cited the size of the visuals as what they enjoyed most about live presentations focused on Earth (Yu 2009). Follow-up oral interviews again mentioned the screen size as an important part of the experience, with multiple respondents highlighting the ease of visibility of the images on the dome compared to a rectangularly-framed presentation in a regular auditorium.
That such large displays are memorable is not surprising. Research comparing televisions with different screen sizes showed that the largest screens increased arousal (the physiological state of being alert, awake, and ready for action), and commanded the greatest attention (Reeves et al. 1999), while action on large screens was rated by viewers as being more intense (Lombard et al. 1997), and movement was viewed as faster, more exciting, and led to a greater sense of physical movement (Lombard et al. 2000). The combination of a physically large display and wide FOV is suspected of leading to greater learning gains in undergraduate students who viewed a live dome astronomy presentation (Yu et al. 2016).
如此大的银幕能够给人留下深刻的印象是很正常的。有研究比较了不同电视屏幕尺寸的影响，结果表明，屏幕最大的电视增加了观众的唤醒状态（即保持警觉和清醒、准备行动的生理状态），并且引起了最大的关注度（Reeves et al. 1999）。观众认为更大的屏幕所呈现的动作更加强烈（Lombard et al. 1997），运动看起来更快、更令人兴奋，并且带来了更强的身体运动感（Lombard et al. 2000）。研究认为，在观看球幕天文学现场演示的本科生中，巨大的银幕和宽广的视场角相结合，能够带来更大的学习收益（Yu et al. 2016）。
A fundamental difference between fulldome films (and its counterpart virtual reality) and traditional cinema is that the latter assumes a rectilinear (also known as linear, geometric, or classical) perspective, where the image at the picture plane represents a single viewing position and direction.
Figure 3: Examples of how projected imagery appear in (a) rectilinear and (b) curvilinear perspective.
图 3：投影的图像在 (a) 直线透视 (b) 曲线透视 下呈现的效果示例
Rectilinear perspective was perfected in the Renaissance with precise mathematical rules for determining the relationship between object size and distance. Artwork created using these rules resulted in objects that appear smaller the further they are from the eye. Objects near the center line-of-sight appear normal in rectilinear perspective. However, as an object is displaced laterally to the left or right, they look distorted because their projections are farther away (and at a greater angle) on the image plane (Fig. 3a, based on the example from Leonardo 1883, p. 63). Leonardo da Vinci was the first to suggest the alternative curvilinear (or synthetic) perspective where instead of a flat projection plane, a curved one is used. In this case (Fig. 3b), the sizes of objects are preserved correctly no matter where they appear. Curvilinear perspective therefore is an attempt to represent the undistorted view that we get when we turn our heads and eyes to examine our environment in all directions. By filming or rendering imagery in a perspective that matches the geometry of the curved screen, fulldome filmmakers are creating a visual experience that mimics the visual experience that we have as we inhabit our real-world environment.
In giant screen cinema, the projected image is so large that the edges of the frame are at the edges of one’s visual field and can effectively “disappear.” Viewers can still turn their heads to see the edges of the giant screen frame, however. The visual experience in fulldome goes even further because even the edges of the dome disappear inside a hemispherical space. Although Shedd (1989, 1999) makes a convincing argument that frames in fulldome do not exist in the same sense as in traditional cinema, we feel that a “frame” can still be defined in a way that is useful for fulldome filmmakers.
As an example, let us use the Denver Museum of Nature & Science’s (DMNS) Gates Planetarium (Fig. 1b), which has raked unidirectional seating and a dome tilted at 25°. Seated audience members’ views of the dome surface depends on the location of their seats and how far back it is reclined. We can define the view as seen by a “reference” theatergoer seated comfortably with head resting against the seat back, gaze naturally pointed forward in a neutral position, with no pronounced head tilt to one side or up or down. This individual will find the combined binocular visual field (using both eyes) to be about 180°x110° (Dragoi 1997; also see Fig. 2).
Figure 4 (top): An audience visibility map for the Gates Planetarium showing what part of the domemaster is visible for an audience in the center seat in the front row (area bound by green line), center row (area bound by yellow line), and back row (area bound by red line). The region shaded red is not visible to even someone in the back row without turning one’s head. The front part of the dome is at the bottom of the image and the back is at the top.
In Fig. 4, we show a fisheye view of the Gates Planetarium dome, with a grid marking out degree increments and major grid lines every 5°. The forward direction of the dome as viewed by the audience is at the bottom of the image, while the back of the dome is at the top of the image. Since the dome is tilted by 25°, the actual zenith is located 25° above the center point of the fisheye.
For an individual seated in the center seat of the front row, the limit of what is visible is bound by the green line and the front edge of the dome in Fig. 4. The front row seats lean back enough to allow content 45° from the zenith to be visible at the top edge of one’s vision. The viewers will have to crane their necks or twist their bodies around to view action that occurs behind the green line. Similarly, people seated in the seats closest to the center of the auditorium will have their view delimited by the yellow line. Because the seats in the center of the house do not lean back as far as the front row seats, the range of visibility is actually quite similar to that of someone from the front row. Finally, the red line shows the edge of what is visible to someone in the back row center, with the rear perimeter running close to the actual zenith of the room.
Figure 5: An audience horizon map for the Gates Planetarium showing the range of horizon tilts that look acceptable to a viewer. Shown as blue lines are two representative horizons: one for a level horizon with zero tilt, and a second where the horizon is tilted up by 25°.
Through experiments at Gates, we discovered that a projected horizon shown in the dome could be within a range of angles away from the viewer’s personal horizon without feeling “wrong” (Yu et al. 2007; also see IMAX Corp. 1999, p. 9). Because the Gates Planetarium is tilted by 25°, the audience’s real-life horizon would cross 25° above the front edge of the domemaster. In Fig. 5, a horizon did not look unrealistic if it was offset (tilted) with respect to the real horizon anywhere inside the blue region. Thus horizons tilted lower down were acceptable, as well as ones tilted up to almost 30° above the viewer’s natural horizon. The perceptual flexibility when deciding whether a horizon looks tilted or not is fortuitous since the apparent tilt of a horizon will also depend on where an audience member is seated in the theater.
The audience visibility and horizon maps in Figs. 4 and 5 came about during the production of the film Black Holes: The Other Side of Infinity (Yu et al. 2007), when versions of them were provided to animators and visualizers as a guide for shot composition. It is specific to the dome tilt and seat lean back angles at the Gates. A map can be evaluated for every seat in the house, but for simplicity’s sake, it was decided to have the boundaries defined by the middle seats in the front, center, and back rows. The lines that circumscribe the audience members’ views of the dome in these rows help determine where the visual “safe area” is located. Visual content must be placed forward of the green line in Fig. 4 in order to be visible to anyone seated in any center seat from the front to the back row without moving their heads. If a visual element is placed outside of the front row safe area, the viewer must be cued to turn and look in a direction that she was not looking before. We will describe ways that this can be done in Part II.
图 4 和图 5 中的观众可见性和地平线分布图都是在制作电影《黑洞：无限的另一面》（Black Holes: The Other Side of Infinity，2006）时绘制的（Yu 等，2007），当时这些示意图供动画师和可视化制作者使用，作为镜头构图的指南。这些示意图是对应于盖茨天文馆特定的球幕倾斜和座椅后仰角度的。虽然可以为影院中的每个座位绘制一张评估图，但为简化起见，决定以前排、中排和后排中间座位定义不同区域的边界。在这些位置观看球幕的观众视野所圈定的线条，有助于确定视觉“安全区”的位置。视觉内容必须放置在图 4 中的绿线前方，以便从前排到后排任何中间座位的观众无需移动头部即可看到。如果视觉元素放置在前排安全区之外，就需要主动提示观众转头查看之前没有观看的方向。我们将在第二部分描述如何实现这一点。
Because Fig. 4 was developed specifically for DMNS’ Gates Planetarium, a corresponding diagram for another dome theater with unidirectional seating will differ in the details, but we would expect it to look similar. For a dome with concentric or epicentric seating, it may not be possible to define a safe area that is equally visible everywhere in the theater outside of the zone around the zenith. For imagery located near the edge of the domemaster, the virtual camera will have to “pan” the visual content so that it “circulates” around the dome in order to be noticed by all audience members. Ideally an individualized visibility grid should be tailored for each theater. But pragmatically, a film producer will likely use just one map to guide content placement for all unidirectional tilted theaters, and a second one for concentric and epicentric theaters.
When filmmakers started working in giant screen cinema, they had to modify the traditional filmmaking techniques they had learned previously. First, they argued that large format films need to be slower paced than traditional cinema. That is, the individual shots need to be longer so that a film would have fewer cuts and total number of shots. IMAX® and OMNIMAX® directors recognized that the frame was so large that viewers needed time to scan the image and take it all in (Tilton 1973; MacGillivray & Freeman 1976). Directors found that IMAX® films could stimulate a viewer’s wonder with views of awe-inspiring natural phenomena, the use of wide vistas, and soaring aerial shots (Crosby 2007).
When the audience is immersed inside a “frameless” flight over a landscape, the camera motion translates into the entire on-screen environment moving. The audience’s view is filled almost entirely with these cues suggesting that they that are moving. Instead of a secondhand experience of viewing the subject of a film from afar, the audience perceives that they are experiencing what is happening on-screen to them in first person (Shedd 1989). When the audience is taken on an aerial flight, the filmed horizon dominates the visual field so any pitching or banking by the camera results in the audience feeling that they and the entire theater are experiencing that motion (Shedd 1999).
The sweeping camera movements can lead to this appearance of movement because of vection, the sensation of self-motion even when the observer is stationary. Experiments show that if viewers see apparent motion in their peripheral vision, the resulting vection produces an illusion of self-motion. Conversely, if motion appears only in the center of the visual field, the subjects feel that only the object in the scene is moving while they remain stationary (Brandt et al. 1973). The generation of vection explains why giant screen cinema and fulldome films can be more involving for the audience.
Unfortunately, this involvement can also cause viewers to develop simulator motion sickness due to the disparity between the audience’s inner ear vestibular systems which suggest they are motionless, and the visuals which give the illusion that the audience is moving. Increasing the rate of motion and widening the FOV so that more of the visual periphery is stimulated are two factors that can worsen motion sickness (LaViola 2000).
Another reason for slowing the action has to do with a visual artifact when objects move too fast on film. An object in motion will appear at different positions in its trajectory in multiple consecutive frames. When projected at normal speed, persistence of vision fuses the individual appearances into the apparent continuous motion of the object. When the screen is too large, however, a moving object will strobe as a series of offset images (Shedd 1998). For giant screen filmmaking, Shedd (1989) recommends that camera pans should move no faster than about 1 degree of arc per second of time in order to minimize this problem. Alternatively, motion blur can be used to smooth over the effect.
需要将动作放慢的另一个原因是，物体在电影中移动过快时会产生视觉伪影（visual artifact）。一个运动中的物体在多个连续帧中会出现在不同的位置上。当以正常速度放映时，视觉暂留（persistence of vision）将这些单独的图像融合成连续运动。然而，当银幕过大时，运动物体会闪烁成一系列错位的图像（Shedd 1998）。对于巨幕电影制作，Shedd（1989）建议摄影机的平移速度不超过约 1 度弧度每秒（1 degree of arc per second），以尽量减少这个问题。此外也可以使用运动模糊来平滑效果。
Cuts and other transitions that lead to new scenes can be disorienting in immersive cinema as well. Shedd (1989) describes the phenomenon of “instant subtraction,” where an object the audience has been looking at suddenly disappears with the onset of a cut. The pacing of the film (how quickly the edits occur) has to be reduced to minimize such subtractions to the audience experience. A film that is cut with a rhythm that looks normal on a normal movie theater screen will appear too fast-paced for the dome. Early OMNIMAX® filmmakers doubled or tripled the length of shots to give viewers time to scan the larger image and adjust to the new scene (Tilton 1973; Wollen 1993).
The above guidelines have been adopted by many fulldome filmmakers. But since fulldome theaters have such wide FOVs, some directors have slowed the camera motion and pacing in their productions even more to avoid motion sickness. The number of shots and cuts in many fulldome films were reduced to a bare minimum. For instance, the first production from the American Museum of Natural History (Passport to the Universe, 2001) consisted of just four individual shots. Camera motion did not disappear despite this. Similar to the soaring aerial shots of IMAX®, virtual flights through space were compelling experiences, and could be used to impart information, such as the graceful spiraling camera motions in Passport to the Universe that conveyed the dimensionality of astrophysical phenomena (Sweitzer 2001).
这些指导原则已被许多全景球幕电影人采用。但由于全景球幕影院的视野非常宽广，一些导演在制作中进一步放慢了摄影机运动和节奏，以避免晕动症。许多全景球幕电影的镜头数量和剪辑次数减少到了最低限度。例如，美国自然历史博物馆的第一部作品《通往宇宙的护照》（Passport to the Universe，2001）仅由四个单独的镜头组成。尽管如此，摄影机运动并未完全消失。与 IMAX® 的高空航拍镜头类似，穿越太空的球幕虚拟飞行是一种引人入胜的体验，并且可以用于传授知识，例如《通往宇宙的护照》中优雅的摄影机螺旋运动就传达了天体物理现象的立体感（Sweitzer 2001）。
Following Shedd’s description of the audience experiencing instead of watching the events on-screen, one can argue that a fulldome cinematic tour is akin to experiencing a journey, rather than merely watching it. For the best audience experience, it follows then that the visual journey should be visually continuous, and the storytelling should be carefully tailored to the virtual trek. There should be a tight integration of narration and visuals, especially since many fulldome shows are about astronomical locales that are unfamiliar to the public. This cumulative experience has been called the “narrative journey” (Wyatt 2005), where the visual tour is augmented by a narration track to create “an intellectual and affective excursion reinforcing the itinerary.”
Pacing of theatrical features, IMAX® 影院长片与 IMAX® 影片的节奏
Despite the convention of reducing the pacing of large screen films, there have been many recent counter-examples to this paradigm. Since 2002, Hollywood movies produced originally in traditional 35-mm film (or its digital equivalent) have undergone Digital Media Remastering (DMR) for exhibition in IMAX® screens (Heuring 2008). Hollywood productions are now even shot natively in digital IMAX® cameras (with the frame cropped for projection in conventional theaters; Lang 2016). How can such fasterpaced feature films be shown and be successful on giant screens, given what we have noted above? Do fast-paced IMAX® films mean that fast-paced fulldome films are also possible?
Figure 6: Average shot lengths of U.S. feature films from 1930-2005 with data from Salt (2009).
图 6：美国 1930-2005 年长片的平均镜头长度（Salt，2009）
To examine this question, we first investigated the pacing of traditional Hollywood feature films, which can be quantified by the average shot length (ASL) of a film, determined by taking the total length of the film and dividing by the number of cuts or edits. Analyses show that Hollywood silent features in the 1920s were cut quite fast, with many having ASLs in the 4-6 sec range (Bordwell 2006, p. 121). With the advent of sound, pacing and camera motion slowed down as equipment proliferated and became bulkier (Bordwell 2006, p. 145). Fig. 6 shows the average ASLs as calculated by Salt (2009) for films from 1930 to 2005.
Starting with a high of more than 11 sec in 1930, the average ASLs decreased by almost a factor of three by the 21st century. There is a bump up in shot lengths during the period 1947-1955 due to the advent of CinemaScope and other widescreen film formats. Directors, producers, and editors slowed down the pace of these films with longer takes. But once it was realized that audiences could handle faster cutting, the ASLs started falling again.
Although the drop in ASLs has been criticized as part of a trend of incoherence in recent blockbuster films (e.g., see the discussion of “chaos cinema” by Stork 2011a, 2011b), analyses of cinematic trends show that film storytelling has not fundamentally changed (Bordwell 2006). Filmic techniques that originally evolved in the 1910s and 1920s are still being used, but in different proportions than that found in the “classical” Hollywood studio period that lasted until the 1960s.
尽管有批评指出 ASL 的下降是近期票房大片电影混乱趋势的一部分（如 Stork 2011a，2011b 关于“混乱电影”的讨论），但对电影发展趋势的分析表明，电影叙事并没有发生根本性的变化（Bordwell 2006）。1910~1920 年代最初发明的电影技巧仍在使用，但其使用比例与 1960 年代以前的“经典”好莱坞制片厂时期影片已经有所不同。
The new style that has evolved in recent decades is called “intensified continuity” by David Bordwell, who cites not only rapid editing, but also the mixed use of wideand narrow-angle lenses, close framings in dialogue scenes, and moving cameras as hallmarks of this modern aesthetic. What is important to note about intensified continuity is that this style is not restricted to just action films. Bordwell points out that starting in the mid-1960s, films of all genres—including dramas, comedies, romances, and musicals—started adopting faster-paced editing, and even directors known for slower paced features in the past started making films with contemporary pacing (Bordwell 2006, pp. 121-124).
The historical pacing of IMAX® films has not been rigorously studied. But in Table 1, we list the ASLs for a number of IMAX® films that we could find in the literature as well as from our own timings (where no footnote is given for a citation). The list includes one of the earliest IMAX® films (North of Superior) as well as the OMNIMAX® film The Dream Is Alive. When compared with Fig. 6, the slowest paced IMAX® films have ASLs that are twice the length of the slowest U.S. feature films from the 1930s and 1950s. However, many giant screen films over the years have ASLs approaching the maximum mean U.S. feature film ASL values of 10-11 sec, including North of Superior and Tropical Rainforest (directed by Ben Shedd). By the late 1990s and the early 2000s, some large screen films have ASLs that drop below 10 sec, approaching average Hollywood feature film ASLs from the 1970s.
IMAX® 电影节奏的历史发展尚未得到系统研究。但在表 1 中，我们列出了一些 IMAX® 电影的 ASL，包括在文献中找到的数据和我们自己统计的数据（未注明出处的为我们自己的数据）。这个列表包括了最早的 IMAX® 电影之一《苏必利尔之北》（North of Superior，1971）以及 OMNIMAX® 电影《执着的梦想》（The Dream Is Alive，1985）。与图 6 相比，节奏最慢的 IMAX® 电影的 ASL 是 1930 年代和 1950 年代节奏最慢的美国长片电影 ASL 的两倍。然而，多年来许多巨幕电影的 ASL 接近美国长片电影的最大 ASL（10-11 秒），包括《苏必利尔之北》和《热带雨林》（Tropical Rainforest，1992，Ben Shedd 导演）。到 20 世纪 90 年代末和 2000 年代初，一些大银幕电影的 ASL 降至 10 秒以下，接近 1970 年代好莱坞长片电影的平均 ASL。
The popularity of films like Pulse: A Stomp Odyssey (6.1 sec ASL) shows that faster-paced giant screen films are possible. Instead of just a few locations, this film ventures out to cover dancers and musicians in locales on five continents. Each set piece contains from one to hundreds of performers. The scenes are edited together from shots from multiple camera set-ups, including close-ups (showing performers shoulders and up), medium (waist and up), and long shots (showing entire bodies); dolly shots where the camera is moving on tracks or on a Steadicam, and crane shots with the camera rising up or down in the air.
的《破铜烂铁2：脉动冲击》（Pulse: A Stomp Odyssey，2002，ASL 为 6.1 秒）等电影的流行表明，快节奏的巨幕电影是可能的。这部电影的取景地点很多，展现了横跨五大洲的舞蹈家和音乐家的表演场景。每个场景中都有一名到数百名表演者，由多机位镜头剪辑而成，包括特写（肩膀以上）、中景（腰部以上）和远景（整个身体），以及推轨镜头（摄影机在轨道或斯坦尼康上移动）和起降镜头（摄影机在空中上升或下降）。
When there are multiple performers, they are lined up across the horizon or massed throughout the screen, meaning there is not just one location the director wants the audience to look at; the viewer has the freedom to shift attention to multiple points on-screen.
The film remains watchable (with a 93% critics score on RottenTomatoes) even when the pacing drops to a quick 4.1 sec ASL (in the sequence with the timbalada drummers in Salvador, Brazil) because the main focus of most shots is the human body, with faces that easily draw attention (for more on this, see Part II). The performers are often in the frame center (or close to it) also making it easy to stay focused on the action after each cut.
即便在节奏加快到 ASL 仅有 4.1 秒的情况下（在巴西萨尔瓦多的 Timbalada 鼓手段落中），这部电影仍然有可看性（在 RottenTomatoes 上获得了 93%的影评人评分），因为大多数镜头的主要焦点是人的身体，聚焦在容易引起注意的面部特征上（更多内容见第二部分）。表演者通常在画面中心或接近中心，这也使得在每次镜头剪辑后观众能够保持对动作的关注。
Nucci (2005) notes that films like Pulse: A Stomp Odyssey goes against the conventional wisdom about how fast editing for giant screen films could be, but also other edicts such as that performers must not appear in a close-up lest their heads end up too high on-screen (Tilton 1973). Nucci believes that since IMAX® films have been dominated by educational films, this genre has set the norm for what is permissible in giant screen filmmaking. Thus nature documentaries are often filled with landscapes which demand stationary shots for viewers to take in vistas, while grand, swooping aerial views of the same also require long shots. This type of slow-paced filmmaking became the prevailing style of giant screen cinema, even though there are not any technical limitations to prevent filmmakers from doing otherwise (Nucci 2005).
Films such as Pulse and NASCAR 3D demonstrated that faster-paced giant screen films could not only be made, but also be popular with audiences as well.1 Although the films in Table 1 have ASLs no faster than the average Hollywood features from the 1970s to the 1980s, they portend the appearance of even faster Hollywood movies that would appear on IMAX® screens courtesy of DMR.
像《破铜烂铁2：脉动冲击》和《超级三维赛车》（NASCAR 3D，2004）这样的电影证明，快节奏的巨幕电影不仅在制作上是可行的，而且也能受到观众欢迎。尽管表 1 中电影的 ASL 没有比 1970~1980 年代好莱坞长片电影的平均 ASL 更快，但它们预示着通过 DMR 转制之后，在 IMAX® 银幕上会出现节奏更快的好莱坞电影。
The Pacing of fulldome films 全景球幕电影的节奏
Table 2: Average shot lengths of a sample of fulldome films.
表 2：一组全景球幕电影样本的平均镜头长度
The Dream Is Alive is the only OMNIMAX® film in Table 1, and is a harbinger of how fast fulldome films could possibly be cut. In fact, the limited listing of fulldome ASLs in Table 2 suggest that many (and perhaps most) fulldome films are much slower paced than even OMNIMAX® films. Even the fastest-paced film in our small sample, Wildest Weather in the Solar System with 21 sec ASL, is only as fast as the giant screen films from the early 1990s, and have shot lengths about twice that of the average Hollywood feature from the 1930s to 1950s. As the preceding analysis shows, there is no technical or perceptual reason why fulldome films need to be as slow as they are. They could but are not cut even as fast as an OMNIMAX® film.
《执着的梦想》是表 1 中唯一的 OMNIMAX® 电影，预示了全景球幕电影可能的剪辑速度。事实上，表 2 中的全景球幕电影 ASL 数据显示，许多（甚至可能是大多数）全景球幕电影的节奏比 OMNIMAX® 电影还要慢。即使是我们所选择的例子中节奏最快的《太阳系中最狂野的天气》（Wildest Weather in the Solar System，2011），其 ASL 为 21 秒，也仅与 20 世纪 90 年代初的巨幕电影相仿，其镜头长度约为 20 世纪 30~50 年代好莱坞长片电影的两倍。前面的分析显示，没有技术上的或感知上的理由表明全景球幕电影需要如此慢的节奏。它们完全可以像 OMNIMAX® 电影那样采用快节奏剪辑，但实际上并没有这样做。
In the end, constructing a film with so few shots is a conscious stylistic or educational choice by a filmmaker. The final two films in Table 2 are from the California Academy of Sciences and are directed by one of the authors (Wyatt). They exemplify the narrative journey approach to storytelling, using continuous camera motions with a minimal number of edits. Habitat Earth consists of eight scenes, while Incoming! has just two long shots.
Many space science fulldome films employ a smooth flight via a single camera maneuvering through planetary and astronomical datasets. This was the style of the early fulldome film Passport to the Universe (2001), which introduced the now standard navigational arc of starting on or near Earth before launching the viewer into deep space. This format was repurposed and reversed for the nonastronomical show Life: A Cosmic Story (2010; Wyatt et al. 2012), which starts in a redwood forest but then zooms into smaller scales until the audience is viewing the animated motions of molecules engaged in photosynthesis inside a plant cell.
许多太空科学主题的全景球幕电影采用单一摄影机平稳飞越行星和天文数据集的方式。早期全景球幕电影《通往宇宙的护照》就采用了这样的风格，该片引入了现在已成标准的“导航叙事弧”（navigational arc），从地球或近地位置开始，逐步将观众带入深空。这种制作方式后来被重用于非天文题材影片《生命：一个宇宙的故事》（Life: A Cosmic Story，2010）（Wyatt 等，2012），该片从红杉森林开始，随后向着更小的尺度缩放，直到观众看到在植物细胞内进行光合作用的分子动画。
But this is also a navigational paradigm intrinsic to planetarium visualization software, where the default real-time flight is a camera zoom similar to what is seen in Ray and Charles Eames’ Powers of Ten film (Emmart 2005). Although the research is limited, the Powers of Ten approach has been shown to be effective for helping students understand scale. The original short film has been shown to be of educational value (Jones et al. 2007), while recreations of the zoom out viewed in a fulldome planetarium result in greater retention and understanding about scale in college students compared to flat screen versions of the same visuals (Yu, Sahami, & Dove 2016).
这同样体现了天文馆可视化软件中一种固有的导航理念，其默认的实时飞行模式是采用相机变焦效果呈现的，类似于 Ray Eames 和 Charles Eames 在电影《十的次方》（Powers of Ten，1977）中采用的手法（Emmart 2005）。尽管相关研究并不广泛，但《十的次方》采用的手法已被证实对于帮助学生理解尺度概念非常有效。原始短片本身已被认为具有教育意义（Jones 等，2007），而在全景球幕天文馆中重现的变焦放大（zoom out）效果，相较于同一视觉效果的平面银幕版本，能够显著提高大学生对尺度概念的理解和记忆（Yu，Sahami，& Dove 2016）。
With fewer cuts and slower pacing, a continuous shot that connects different size domains allows audiences to see the relationships and nested contexts between natural phenomena. This is especially important in astronomy, where the content is beyond the ordinary everyday experience of the general public. Having this additional context can be important for promoting understanding.
Even for live fulldome lectures focused on Earth topics, audience members cited zooming and seeing details at multiple geographic scales as something they liked best about the presentation in the planetarium (Yu 2009). Such transitions place local phenomena within a global context, which is important when teaching about interconnected Earth systems where global environmental change phenomena can manifest itself in different ways at different scales (Yu et al. 2013). Finally, such unbroken transitions may help make a visceral connection to the viewer. As one of us has written (Wyatt 2014),
… transitions in scale are particularly amenable to the immersive environment, allowing viewers to experience continuous changes in size relationships that helps in constructing mental models of the phenomena. Thus, when we depict human-scaled phenomena in a fulldome planetarium—and then continuously transition to larger or smaller scales—we have an opportunity to connect spatial relationships that include our own human, embodied experience …
[A shot that transitions through multiple scales] enables viewers to link the objects and the concepts in a coherent, unified fashion. It establishes context for the viewer in a highly visual, intuitive, and visceral manner.
Traditional film has built up a repertoire of cinematic language that describes the fundamental elements of shots or scenes, and this language can be used by giant screen and fulldome filmmakers. However, as Ben Shedd has pointed out (1989) and as described in §4, another paradigm exists where the film frame disappears and what is shown can be thought of as an environment that the audience feels is personally experiencing. Here, we describe a theoretical framework where we assume as given the fundamental axiom, Action is happening to the audience, instead of being viewed on-screen by the audience. We can then ask, how do we re-frame cinematic language for fulldome so that the vocabulary is centered around the audience experience?
传统电影已经发展出一套描述镜头和场景基本元素的电影语言，这套语言可以被巨幕和全景球幕电影人所使用。然而，正如 Ben Shedd（1989）以及第 4 节所述的那样，还存在另一种范式，在这种范式中电影的边框消失，影片所呈现的内容被观众视为他们亲身体验的环境。在这里我们描述一个理论框架，其中假设一个基本公理：对观众而言，动作是发生在自己身上的，而不是在银幕上观看的。由此我们可以提问，如何重新定义全景球幕电影的电影语言，使其相关术语能够围绕观众体验展开？
Table 3: A list of types of camera shots and movements in traditional cinema with their complements in our theory of immersive cinema.
表 3：传统电影中的摄影机镜头和运动术语，及其在全景球幕电影中的对应术语
We propose a short list of terms from traditional cinema in Table 3 with their corresponding re-framings in immersive cinema. The list above is not meant to be complete, but is intended to start a dialogue for others interested in exploring this model and re-thinking these basic filmmaking concepts. We stress that this theoretical approach to immersive cinema is neither the only filmmaking style that can be applied to fulldome film, nor even what is recommended in some situations. Our earlier discussion of Pulse: A Stomp Odyssey shows that there are no absolute rules that apply equally in all situations for giant screen film. But by organizing the results from this assumption of audience immersion, we propose a common vocabulary to promote further discussion and exploration. Filmmakers can help confirm or invalidate elements of this theory with examples from their films, and researchers can do the same with audience studies.
Based on our fundamental axiom of immersive cinema, the audience occupies the same environment as the characters in the film, and distinctions erode between shots that are objective (distant and independent) and subjective (personal point of view). The audience member becomes a participant, and it follows then that immersive filmmaking is inherently a subjective experience. Hence, while static camera setups in traditional cinema that use lenses to control the visual field of what is visible on a framed screen, terms such as “close up” and “wide” shots describe how much the object of interestoften a person—fills the screen.
For instance, a close up, when in reference to an actor, describes a shot in which the actor is visible in frame from the shoulders to the top of the head (e.g., Katz 1991, p. 122). In immersive cinema, there is no frame to help define what type of shot has been set up. Although fulldome content can capitalize on small variations in the content’s angle of view (AOV) without excessive distortion, the medium cannot tolerate the dramatic changes in camera AOV that traditional cinema frequently employs. Thus designations such as “close up” are highly subjective, especially for viewers in different parts of the theater. Instead, we propose describing shots in terms of the physical location and movement of the camera (and by proxy, the viewer) within the filmed scene.
When an audience moves away from an on-screen subject, that subject appears smaller as more of the surroundings are revealed, creating the appearance of a wider shot. This is not done by an optical zoom but by moving the audience viewpoint from one location to another. Thus a wide shot or establishing shot is a viewpoint that is far away from the subject, revealing more of the scenery around the subject. In a close-up, the subject has a larger AOV, and fills up more of the visual field.
Camera motions have an exact correspondence to the perceived audience motion in immersive cinema. Tracking (or dollying) in and out is equivalent to the audience moving forward and backward; tracking left and right translates into lateral motion for the audience. Crane shots that allow a camera to shift vertically give the viewer a sensation of upward or downward motion. Camera pans to the left and right or tilts up and down result in similar changes in orientation for an audience member.
A cut occurs when a scene transitions from one camera placement to a different one in either the same scene or a completely new one. In immersive cinema, this results in an instant teleportation of the audience into the new location. If the new scene is distinctly different from the old one, it may take a second or more for a viewer to orient himself. The commonly used technique of cutting between two opposite viewpoints in a dialogue scene (commonly called a reverse angle shot) becomes jarring when considered in the context of immersive cinema. A compromise technique may be to “orbit” the camera around the two subjects, so that a continuous frame of reference is maintained, as the camera (and audience) circumnavigates through different points of view of the actors. Similarly, a series of edited shots can be replaced with a single unbroken shot. Through camera movement along a carefully-devised camera path, the visual story develops by what is visible or is revealed to the audience as they are taken on this journey.
How can we use this theory of immersive cinema to make sense of how fulldome film works? Let us take the last example of a continuous camera path that takes an audience through a scene. This camera path defines the audience experience in narrative journey-style fulldome films. If the camera travels linearly along a path and there are no edits to instantaneously transport the viewer from one camera point-of-view to another, it would be difficult for the viewer to be surprised by anything within the scene. The slower camera motions that are necessary in fulldome film means that a new element that appears in the distance will gradually grow in size as the camera approaches it. Unless the object appears from behind the audience or just beyond the edge of the dome (spring line), it will be hard to miss.
A way around this is a dedicated reveal. Here a foreground element or the edge of the screen eclipses something further in the background. Some combination of camera or object movement can result in the object emerging and revealing itself to surprise the audience. One such example can be found in Peter Popp’s Realm of Light: A Brief History of Life (2009), where a cluster of dark asteroids float in the foreground, obscuring something glowing in the distant background. The audience is attracted by the mysterious light which is revealed as the camera (and the audience) navigate through the floating debris field, slowly exposing a giant red star looming in the distance (Fig. 7).
解决这个问题的一种方法是专门的“揭示”（reveal）手法。将后景物体通过前景元素或银幕边缘遮挡起来，随后结合摄影机或物体的运动，让后景物体逐渐显现出来，可以给观众带来惊喜感。Peter Popp 执导的影片《光之领域：生命的简史》（Realm of Light - A Brief History of Life，2009）就应用了这一手法，片中一簇黑暗的小行星漂浮在前景中，遮挡住远处后景中发光的物体。观众被神秘的光线所吸引，当摄影机（和观众）穿越漂浮的小行星碎片场域时，远处巨大的红巨星逐渐揭示出来（图 7）。
In many cases, it may be difficult or impossible to create a single continuous shot that includes all of the scenes shot or created for a film. It will be necessary to join together a mix of disparate shots. If we follow our theory of immersive cinema, we want to minimize disruptions to the audience experience. Multiple solutions showing how this can be done can be found in Android Jones’s Samskara (2016), an art film inspired by Vedic myth and Sanskrit mantras, consisting of flight through labyrinthine computer-generated scenes populated with densely detailed imagery. Although the visuals appear to consist of one long, continuous shot, they are actually constructed from multiple scenes connected by subtle dissolves and wipes. There are edits throughout the film where an entire environment can change without the viewer immediately noticing that it is happening.
在许多情况下，很难（或无法）创建一个包含所有场景的连续镜头，需要将不同的镜头组合在一起。如果我们遵循沉浸式影院理论，则会希望尽量减少对观众体验的干扰。Android Jones 执导的影片《生生不息》（Samskara，2016）展示了多种解决方案，这是一部受吠陀神话和梵文咒语启发的艺术电影，由计算机生成的复杂图像场景组成了迷宫般的飞行体验。虽然影片的视觉效果似乎是由一个很长的连续镜头构成，但实际上是由多个场景通过细微的溶解（dissolves）和擦除（wipe）转场连接而成。电影中有许多次镜头剪辑，通过上述手法可以在观众不易察觉的情况下完全转换场景。
Our first example is at the end of a scene with the camera moving past the blue-gray feather-adorned headdress of a monolithic figure, with several orange, hollow polyhedral shapes further ahead, and a dense background of texture and detail everywhere else (Fig. 8a). As the camera passes by the orange polyhedra, they and the rest of the scene darken (Fig. 8b). A deep red-magenta trilobal pattern fades up in the center of the viewer’s visual field (Fig. 8c), and grows as it animates with a rapid pulse. Since attention is drawn to this dominating feature, the viewer is less apt to notice the change elsewhere on the dome, as the original background darkens and a dissolve leads to a new starry background (Fig. 8d). The animating pattern quickly fills up the entire screen, so the new background is covered up before the viewer has a chance to reflect on it.
A wipe is a film transition where a new shot replaces an old one by traveling across the frame. These tend to be noticeable if the two shots are very different looking. However, Samskara has subtle wipes by objects, where a foreground object moving through the shot helps hide the effect. In Fig. 9a, the camera is headed towards a point above the reddishorange head of another giant figure. Two butterflies appear high up and off to the left and right, meaning they are in the viewer’s peripheral vision if she is looking forward towards the direction of motion. (See the two series of overlapping wings emerging in Fig. 9b.) Other butterflies flit through the shot, but only those two get close enough to the virtual camera to momentarily fill up the entire domemaster frame (Fig. 9c). Their movement provides cover for the background scene to disappear and be replaced by a new one (Fig. 9d).
Another wipe by foreground object helps hide a jump between two diametrically opposite camera angles near the end of the film. In this scene, the camera has been pirouetting around a demonic head that is lit orange, complementary to the deep blue that dominates the shot. The environment is filled with geometrical blocks consisting of skyscraper-like shapes, including spires that resemble the Empire State Building. In Fig. 10a, a foreground element consisting of skyscraper blocks emerges from below the bottom of the frame, just to the right of the head. The camera tracks left as the foreground mass of skyscrapers grows and also moves left. The foreground skyscrapers, now grown to a triangular mass, shift to completely block our view of the towering head, with only its red tongue still not completely cut off on the left in Fig. 10b.
A wipe to a new scene starts in Fig. 10c, with the shape of the wipe matching the boundary of the triangle of skyscrapers. Notice the new background building appearing where the tongue was in Fig. 10b, and the mirrored bright clump of the Milky Way to the right of the skyscraper triangle in the incoming shot, while the bright clump from the previous shot is still visible in the back of the dome (top of the domemaster frame). The wipe grows to encompass the rest of the shot; the background building seen earlier is now revealed to be the Empire State Building, the top of it glowing blue against the Milky Way (Fig. 10d).
As the camera continues to track left in Fig. 10e, the triangular block of buildings in the foreground moves off to the left, and we see that the camera is now on the opposite side of where it started relative to the demonic head in Fig. 10a. The position of the Milky Way has also flipped in the sky. Throughout this transition, the triangular block of skyscrapers dominates the shot with its size, central position, and motion, drawing the viewer’s attention, even as the rest of the scene around it completely changes.
当摄影机在图 10e 中继续向左移动时，前景的三角形建筑块移到左侧，我们看到相对于图 10a 中的恶魔头部，现在摄影机已经位于对面了，银河在天空中的位置也翻转了。在整个转场过程中，前景的三角形块状摩天大楼以其大小、中心位置和运动占据主导地位，吸引观众的注意力，掩盖了场景其余部分的完全改变。
Coming next 下篇预告
This ends Part I of our two part paper. In the next issue, we look further into how a fulldome director can construct shots and create compositions that are clear to the audience. We will show examples from additional films highlighting techniques to focus a viewer’s attention even when she theoretically has the freedom to look anywhere within the hemispherical display.
We thank Peter Popp and Android Jones for permission to use images from their films. We also thank Kathy Honda for help with our library research.
感谢 Peter Popp 和 Android Jones 允许我们使用他们电影中的静帧，感谢 Kathy Honda 帮助我们进行图书馆文献搜索。
References 参考文献
Block, B. 2008, The Visual Story, 2nd edition, Burlington, MA: Focal Press.
Bordwell, D. 2006, The Way Hollywood Tells It: Story and Style in Modern Movies, Berkeley, CA: University of California Press.
Brandt, T., Dichgans, J., & Koenig, E. 1973, “Differential effects of central versus peripheral vision on egocentric and exocentric motion perception,” Experimental Brain Research, 16(5), 476-491.
IMAX Corp. 1999, IMAX: The 15/70 Filmmaker’s Manual.
Jones, M.G., Taylor, A., Minogue, J., Broadwell, B., Wiebe, E., & Carter, G. 2007, “Understanding Scale: Powers of Ten,” Journal of Science Education and Technology, 16(2), 191-202.
Jukic, S. 2016 April 5, “4K & Ultra HD (UHD) Resolution – Comparison of 4k vs 1080p and Everything You Need to Know,” https://4k.com/resolution/.
Katz, S. 1991, Film Directing Shot by Shot: Visualizing from Concept to Screen, Focal Press.
LaViola Jr, J. J. 2000, “A discussion of cybersickness in virtual environments,” ACM SIGCHI Bulletin, 32(1), 47-56.
Leonardo da Vinci, 1883, The Literary Works of Leonardo da Vinci Compiled and Edited from the Original Manuscripts, J.P. Richter (ed.), London: Sampson Low, Marston, Searle & Rivington.
Lombard, M., Ditton, T. B., Grabe, M. E., & Reich, R. D. 1997, “The role of screen size in viewer responses to television fare,” Communication Reports, 10(1), 95-106.
Lombard, M., Reich, R. D., Grabe, M. E., Bracken, C. C., & Ditton, T. B. 2000, “Presence and television: The Role of Screen Size,” Human Communication Research, 26(1), 75-98.
MacGillivray, G. and Freeman, J. 1976 July, “Producing the IMAX Motion Picture: ‘To Fly,’” American Cinematographer, 57(7), 750-753, 792.
n.a. 2016 March/April, “The World’s First True8K TM Dome Theater: The Ultimate Planetarium Experience,” HMNS News, 12-15.
Nucci, M. 2010, Large Format and Mediation of the Natural World: Vision, Technology and the Sublime, PhD thesis, Rutgers, State University of New Jersey, New Brunswick, New Jersey.
Reeves, B., Lang, A., Kim, E. Y., & Tatar, D. 1999, “The effects of screen size and message content on attention and arousal,” Media Psychology, 1(1), 49-67.
Salt, B. 2009, “The shape of 1959,” New Review of Film and Television Studies, 7(4), 393-409.
Schatz, T. 1996, The Genius of the System: Hollywood Filmmaking in the Studio Era. New York: Henry Holt and Co.
Shaw, W. C., & Douglas, J. C. 1983, “IMAX® and OMNIMAX ® Theatre Design,” SMPTE Journal, 92(3), 284-290.
Sweitzer, J. S. 2001, “Cosmic Spaceflight 101: Lessons I Learned Flying to the Edge of the Observable Universe,” Planetarian, 30(4), 4-8, 21.
Thompson, K. and Bordwell, D. 2009, Film History: An Introduction, 3/e, New York: McGraw-Hill.
Tilton, R. 1973 August, “Filming ‘Garden Isle’ in the World’s Largest Film Format,” American Cinematographer, 1026-1027. 1062-1063.
Wollen, T. 1993, “The bigger the better: From CinemaScope to Imax,” Future visions: New technologies of the screen, 10-30.
Wyatt, R. 2005, “Planetarium Paradigm Shift,” Planetarian, 34(3), 15-19.
Wyatt, R. 2014 May/June, “Planetariums as 21st Century Digital Dioramas,” Informal Learning Review, 126, 3-8.
Wyatt, R., Kennedy, T., Lapré, J., Schmitt, M. 2012, “The Making of Life: A Cosmic Story,” Planetarian, 41(1), 16-20.
Yarbus, A. L. 1967, Eye Movements and Vision, New York, NY: Plenum Press.
Yu, K.C. 2009, “Digital Planetariums for Geology and Geography Education,” Planetarian, 36(3), 6-12, 64.
Yu, K.C., Brownell, M., Schoemer, J., Neafus, D., Lucas, T. & Zager, Z. 2007, “Live Action Film Footage for an Astronomy Fulldome Show,” Planetarian, 36(1), 6–17, 86–87.
Yu, K.C., Connolly, R., Gardiner, N., Hamilton, H., Koontz, K., McConville, D., Sickler, J., & Wyatt, R. 2013, “The Worldviews Network: Digital Earth for Ecological and Resilience Education,” 8th International Symposium on Digital Earth, 26-29 Aug 2013, Kuching, Sarawak, Malaysia.
Yu, K.C., Sahami, K., Denn, G., Sahami, V., & Sessions, L. 2016, “Immersive Planetarium Visualizations for Teaching Solar System Moon Concepts to Undergraduates,” Journal of Astronomy & Earth Science Education, in press.
Yu, K.C., Sahami, K., & Dove, J. 2016, “Learning About Scale of the Solar System Using Digital Planetarium Visualizations,” American Journal of Physics, in review.
Zettl, H. 1998, Sight, Sound, Motion: Applied Media Aesthetics, 3/e. Belmont, CA: Wadsworth Publishing.
Gamma characterizes the reproduction of tone scale in an imaging system. Gamma summarizes, in a single numerical parameter, the nonlinear relationship between code value – in an 8-bit system, from 0 through 255 – and luminance. Nearly all image coding systems are nonlinear, and so involve values of gamma different from unity.
Owing to poor understanding of tone scale reproduction, and to misconceptions about nonlinear coding, gamma has acquired a terrible reputation in computer graphics and image processing. In addition, the world-wide web suffers from poor reproduction of grayscale and color images, due to poor handling of nonlinear image coding. This paper aims to make gamma respectable again.
The left-hand column in this table summarizes the allegations that have led to gamma’s bad reputation. But the reputation is ill-founded – these allegations are false! In the right column, I outline the facts:
左栏总结了导致伽马名声不好的指控。但这些指控是不成立的——它们是错误的！在右栏中，我列出了事实：
Misconception 误解
Fact 事实
A CRT’s phosphor has a nonlinear response to beam current. CRT 的荧光粉对电子束电流有非线性响应。
The electron gun of a CRT is responsible for its nonlinearity, not the phosphor. CRT 的非线性是由电子枪引起的，而不是荧光粉。
The nonlinearity of a CRT monitor is a defect that needs to be corrected. CRT 显示器的非线性是需要纠正的缺陷。
The nonlinearity of a CRT is very nearly the inverse of the lightness sensitivity of human vision. The nonlinearity causes a CRT’s response to be roughly perceptually uniform. Far from being a defect, this feature is highly desirable. CRT 的非线性几乎是人类明度敏感度（lightness sensitivity）的反函数，这种非线性使 CRT 的响应大致是感知上均匀的（perceptually uniform）。这不仅不是缺陷，反而是一个非常理想的特性。
The main purpose of gamma correction is to compensate for the nonlinearity of the CRT. Gamma 校正的主要目的是补偿 CRT 的非线性。
The main purpose of gamma correction in video, desktop graphics, prepress, JPEG, and MPEG is to code luminance or tristimulus values (proportional to intensity) into a perceptually-uniform domain, so as optimize perceptual performance of a limited number of bits in each RGB (or CMYK) component. 在视频、桌面图形、印前处理、JPEG 和 MPEG 中，伽马校正的主要目的是将（与强度成比例的）光亮度或三刺激值编码到感知均匀的域中，以优化每个 RGB（或 CMYK）分量中有限位数的感知性能。
Ideally, linear-intensity representations should be used to represent image data. 理想情况下，应该使用线性强度表示法来表示图像数据。
If a quantity proportional to intensity represents image data, then 11 bits or more would be necessary in each component to achieve high-quality image reproduction. With nonlinear (gamma-corrected) coding, just 8 bits are sufficient. 如果使用与强度成比例的量来表示图像数据，那么每个分量需要 11 位或更多位，才能实现高质量的图像再现。而使用非线性（经过伽马校正的）编码，只需 8 位就足够了。
A CRT is characterized by a power function that relates luminance L to voltage V’： CRT 的特性是光亮度 L 与电压 V’ 之间的幂函数关系：。
The exponent γ varies anywhere from about 1.4 to 3.5. 指数 的值变化范围在 1.4 到 3.5 之间。
The exponent itself varies over a rather narrow range, about 2.35 to 2.55. The alleged wide variation comes from variation in offset term of the equation, not the exponent: Wide variation is due to failure to correctly set the black level. 指数本身的变化范围相当狭窄，大约在 2.35 到 2.55 之间。所谓的广泛变化来自方程中的偏移项（offset term）变化，而不是指数：变化范围较大是由于未能正确设置黑电平。
Gamma correction is accomplished by inverting this equation. 伽马校正是通过对上述方程取反实现的。
Gamma correction is roughly the inverse of this equation, but two alterations must be introduced to achieve good perceptual performance. First, a linear segment is introduced into the transfer function, to minimize the introduction of noise in very dark areas of the image. Second, the exponent at the encoder is made somewhat greater than the ideal mathematical value, in order to impose a rendering intent that compensates for subjective effects upon image display. 伽马校正大致是这个方程的逆过程，但为了实现良好的感知性能，必须引入两个改动。首先，在转换函数中引入一个线性段，是为了将图像极暗部引入的噪声最小化。其次，编码器的指数值被设定得比理想数学值略大，以补偿图像显示时的渲染意图（rendering intent）。
CRT variation is responsible for wide variability in tone scale reproduction when images are exchanged among computers. 由于 CRT 的变化，计算机之间交换图像时色调再现的变化很大。
Poor performance in image exchange is generally due to lack of control over transfer functions that are applied when image data is acquired, processed, stored, and displayed. 图像交换中的性能差异通常是由于在图像数据获取、处理、存储和显示时缺乏对转换函数的控制。
Macintosh monitors have nonstandard values of gamma. Macintosh 显示器的伽马值是非标准的。
All CRT monitors, including those used with Macintosh computers, produce essentially identical response to voltage. But the Macintosh QuickDraw graphics subsystem involves a lookup table that is loaded by default with an unusual transfer function. It is the default values loaded into the lookup table, not the monitor characteristics, that impose the nonstandard Macintosh gamma. 所有 CRT 显示器，包括用于 Macintosh 计算机的显示器，对电压的响应基本相同。但 Macintosh 的 QuickDraw 图形子系统（译者注：QuickDraw 是苹果公司开发的一种图形库，用于处理图形和图像）涉及一个查找表，该表默认加载了一个不寻常的转换函数，该函数是加载到查找表中的默认值，而不是显示器特性，这导致了 Macintosh 的伽马是非标准的。
Gamma problems can be circumvented by loading a lookup table having a suitable gamma value. 通过加载具有合适伽马值的查找表可以规避伽马问题。
Loading a particular lookup table, or a particular value of gamma, alters the relationship of data in the frame buffer to linear-light “intensity” (properly, luminance, or tristimulus value). This may have the intended effect on a particular image. However, loading a new lookup table will disturb the code-to-luminance mapping that is assumed by the graphics subsystem, by other images, or by other windows. This is liable to alter color values that are supposed to stay fixed. 加载特定的查找表或特定的伽马值，会改变帧缓冲器中数据与线性光“强度”（准确地说是光亮度或三刺激值）之间的关系，这可能对特定图像产生预期的效果。然而，加载新的查找表会扰乱图形子系统、其他图像或其他窗口假设的“码值到光亮度”的映射，这可能会改变本应保持固定的颜色值。
Macintosh computers are shipped from the factory with gamma set to 1.8. SGI machines default to gamma of 1.7. To make an SGI machine display pictures like a Mac, set SGI gamma to 1.8. Macintosh 计算机出厂时设置的伽马值为 1.8。SGI 机器的默认伽马为 1.7。要让 SGI 机器显示的图像像 Mac 一样，需要将 SGI 的伽马设置为 1.8。
On the Macintosh, setting a numerical gamma setting of g loads into the framebuffer’s lookup table a power function with the exponent g ⁄ 2.61 . On an SGI, setting a numerical gamma setting of g loads into the lookup table a power function with the exponent 1 ⁄ g . To make an SGI machine behave like a Mac, you must set SGI gamma to 1.45. 在 Macintosh 上，设置伽马数值为 g 会将幂函数加载到帧缓冲器的查找表中，其指数为 g/2.61。在 SGI 上，设置数值伽马值 g 会将幂函数加载到查找表中，其指数为 1/g。要使 SGI 机器的表现像 Mac，需要将 SGI 的 Gamma 设置为 1.45。
Gamma problems can be avoided when exchanging images by tagging every image file with a suitable gamma value. 交换图像时，通过为每个图像文件标记一个合适的伽马值可以避免伽马问题。
Various tag schemes have been standardized; some tags are coded into image files. However, application software today generally pays no attention to the tags, so tagging image files is not helpful today. It is obviously a good idea to avoid subjecting an image file to cascaded transfer functions during processing. However, the tag approach fails to recognize that image data should be originated and maintained in a perceptually-based code. 各种标签方案已经标准化，有些标签编码在图像文件中。然而，现今的应用软件通常不会注意这些标签，因此对图像文件进行标记在今天并没有帮助。显然，在处理过程中避免对图像文件应用级联转换函数（cascaded transfer functions）是个好主意。然而，标签方法未能认识到图像数据应在感知基础上的编码中生成和维护。
JPEG compresses RGB data, and reproduces RGB data upon decompression. The JPEG algorithm itself is completely independent of whatever transfer function is used. JPEG 压缩 RGB 数据，并在解压时重现 RGB 数据。JPEG 算法本身完全独立于所使用的任何转换函数。
JPEG and other lossy image compression algorithms depend on discarding information that won’t be perceived. It is vital that the data presented to a JPEG compressor be coded in a perceptually-uniform manner, so that the information discarded has minimal perceptual impact. Also, although standardized as an image compression algorithm, JPEG is so popular that it is now effectively an image interchange standard. Standardization of the transfer function is necessary in order for JPEG to meet its users’ expectations. JPEG 和其他有损图像压缩算法依赖于丢弃不会被感知的信息。因此，向 JPEG 压缩器提供的数据必须以感知均匀的方式编码，以便丢弃的信息对感知影响最小。此外，尽管 JPEG 被标准化为图像压缩算法，但由于其使用的广泛性，它现在实际上是图像交换标准。为了使 JPEG 满足用户的期望，转换函数的标准化是必要的。
Intensity 强度
Intensity is the rate of flow of radiant energy, per unit solid angle – that is, in a particular, specified direction. In image science, we measure power over some interval of the electromagnetic spectrum. We’re usually interested in power radiating from or incident on a surface. Intensity is what I call a linear-light measure, expressed in units such as watts per steradian.
强度是每单位立体角内辐射能量的流速，也就是特定方向上的辐射强度。在图像科学中，我们测量电磁光谱某个区间内的功率，通常关注从表面发出的或落在表面上的辐射功率。强度是对线性光的一种度量，用瓦特每立体角（watts per steradian）表示。
The CIE has defined luminance, denoted Y, as intensity per unit area, weighted by a spectral sensitivity function that is characteristic of vision. The magnitude of luminance is proportional to physical power; in that sense it is like intensity. But the spectral composition of luminance is related to the brightness sensitivity of human vision.
国际照明委员会（CIE）定义了光亮度，用 Y 表示，作为每单位面积的强度，并按视觉特性的光谱灵敏度函数加权。光亮度的大小与物理功率成正比，在这方面它与强度类似。但光亮度的光谱组成与人类视觉的视亮度敏感度（brightness sensitivity）有关。
Luminance can be computed as a properly-weighted sum of linear-light (tristimulus) red, green, and blue primary components. For contemporary video cameras, studio standards, and CRT phosphors, the luminance equation is this:
Video equipment forms a luma component Y’ as a weighted sum of nonlinear R’G’B’ primary components. The nonlinear quantity is often incorrectly referred to as luminance by video engineers who are unfamiliar with color science.
The luminance generated by a physical device is usually not proportional to the applied signal – usually, there is a nonlinear relationship. A conventional CRT has a power-law response to voltage: Luminance produced at the face of the display is approximately the applied voltage raised to the five-halves power. The numerical value of the exponent of this power function, 2.5, is colloquially known as gamma. This nonlinearity must be compensated in order to achieve correct reproduction of luminance. An example of the response of an actual CRT is graphed, at three settings of the CONTRAST control, in Figure 1 above. [1]
Figure 1 CRT’s transfer function is shown at three different settings of the CONTRAST (or PICTURE) control. Here I show CONTRAST altering the y-axis (luminance) scaling; owing to the properties of a power function, scaling the x-axis (video signal) has an equivalent effect. The graph indicates a video signal having a voltage from zero to 700 mV. In a typical eight-bit digital-to-analog converter in a computer graphics subsystem, black is at code zero, and white is at code 255.
图 1 显示了在三种不同对比度（CONTRAST 或 PICTURE）设置下的 CRT 转换函数。这里展示了对比度设置的变化如何对 y 轴（光亮度）进行缩放；由于幂函数的性质，缩放 x 轴（视频信号）也会产生相同的效果。图表展示了电压从零到 700 毫伏的视频信号。在典型的计算机图形子系统的 8 位数模转换器中，黑色对应码值零，白色对应码值 255。
It is alleged that the power function exponent γ of a CRT varies over a wide range; values as low as 1.4 and as high as 3.5 are cited in the literature. The graphs and captions in Figures 4 and 6 opposite show that wide variation in the apparent gamma value will result if the monitor’s BLACK LEVEL (or “BRIGHTNESS”) control is improperly adjusted. [2]
At a particular level of adaptation, human vision responds to about a hundred-to-one contrast ratio of luminance from white to black. Within this range, vision has a nonlinear response to luminance: Lightness perception is roughly logarithmic. A source having a luminance only 18% of a reference luminance appears about half as bright. The perceptual response to luminance is called Lightness. Vision researchers have modeled lightness sensitivity with various mathematical functions, as shown in Figure 2 below.
Figure 2 Luminance and lightness. The relationship between lightness-scale value V and luminance factor Y is plotted in accordance with different formulae. Redrawn from Fig. 2 (6.3) from Wyszecki and Stiles, Color Science (New York: Wiley, 1982).
图 2：光亮度和明度。 根据不同公式绘制了明度值 V 与光亮度因子 Y 之间的关系。图表重新绘制自 Wyszecki 和 Stiles 的《色彩科学》（纽约：Wiley, 1982）中的图 2（6.3）。
The CIE has adopted a standard function L* (pronounced “EL-star”), defined as a modified cube root: [3]
国际照明委员会（CIE）采用了一种标准函数 L*（读作“EL-star”），定义为修正后的立方根：
Y n is the luminance of the white reference. A linear segment with a slope of 903.3 is applied near black. L* has a range of 0 to 100. A unit change in L* is taken to be approximately the threshold of visibility. In other words, you can detect a difference between intensities when the ratio between them is greater than about one percent.
Figure 3 BRIGHTNESS control has the effect of sliding the black-to-white video signal scale left and right along the 2.5-power function of the display. Here, BRIGHTNESS is set too high; a significant amount of luminance is produced at zero video signal level. No video signal can cause true black to be displayed, and the picture content rides on an overall pedestal of gray. Contrast ratio is degraded.
Figure 4 Gamma 3.5 A naive approach to the measurement of CRT nonlinearity is to model the response as L = (V’) γ , and to find the exponent of the power function that is the best fit to the voltage-to-intensity transfer function of a particular CRT. However, if this measurement is undertaken with BRIGHTNESS set too high, an unrealistically large value of gamma results from the modelled curve being “pegged” at the origin.
Figure 5 BRIGHTNESS control set too low causes a range of input signal levels near black to be reproduced “crushed” or “swallowed,” reproduced indistinguishably from black. A cinematographer might refer to this situation as “lack of details in the shadows,” however, all information in the shadows is lost, not just the details.
Figure 6 Gamma 1.4 If the transfer function is modelled as L = (V’) γ with Black Level set too low, an unrealistically small value of gamma results. However, if the transfer function is modeled with a function of the form L = (V’+ ε ) 2.5 that accommodates black level error, then a good fit is achieved. Misintepretations in the measurement of CRT nonlinearity have led to assertions about CRTs being highly unpredictable devices, and have led to image exchange standards employing quite unrealistic values of gamma.
Suppose that you wish to convey luminance values of an image through a channel having a few hundred or a few thousand discrete levels. Consider linear light coding, sketched in the margin, where code zero represents black. No matter what code is at the top end, code 100 represents a shade of gray that lies approximately at the perceptual threshold. For codes below 100, the ratio of intensities between adjacent code values is greater than 1 percent. At code 25, the ratio between adjacent codes is 4 percent. In smooth-shaded regions of an image, the luminance difference between adjacent code values, such as between code 25 and code 26, will cause visible banding or contouring. For codes above 100, the ratio of luminance values between adjacent codes is less than 1 percent: Code 201 is perceptually useless, and could be discarded without being noticed. In an 8-bit system, the highest code value – the brightest white – is at code 255. In an 8-bit linear-light system, the ratio between the brightest white and the darkest grey that can be reproduced without contouring is a mere 2.55:1.
Figure 7 Linear light coding: the code 100 problem
图 7：线性光编码的码值 100 问题
To avoid perceptible steps at the black end of the scale, it is necessary to have coding that represents different luminance levels 1.00, 1.01, 1.02, and so on. If linear light coding is used, an absolute “delta” of 0.01 must be maintained all the way up the scale to white. To encompass the 100:1 luminance range vision of requires about 9900 codes, or about 14 bits for each of the R, G, and B components of the image.
If you use nonlinear coding, then the 1.01 “delta” required at the black end of the scale applies as a ratio – not as an absolute increment – and progresses like compound interest up to white. This results in about 463 codes, or about nine bits per component. Eight bits, nonlinearly coded according to Rec. 709, is sufficient for broadcast-quality digital television at a contrast ratio of about 50:1. [4]
In computer-generated imagery (CGI), linear-light coding is typically used in the frame buffer, as sketched in Figure 8 above. Often only 8 bits are provided in the framebuffer. When luminance data traverses the 8-bit Bottleneck indicated in the sketch, serious contouring results.
Figure 8 Gamma in computer-generated imagery (CGI)
图 8：计算机生成影像（CGI）中的伽马
Video coding 视频编码
To code luminance into a small number of steps, say 256, then the codes should be assigned to intensities according to the properties of perception, in order for the most effective perceptual use to be made of the available codes. A transfer function similar to the lightness sensitivity of vision should be imposed at encoding.
A CRT’s response is very nearly the inverse of the lightness sensitivity of vision: When image data is coded for perception at the encoder – for example, by the Rec. 709 transfer function graphed in Figure 9 above the coding is inverted by the CRT, without the necessity to dedicate any circuitry to the task. The fact that a CRT’s transfer function is very nearly the inverse of the lightness sensitivity of vision is an amazing, and fortunate, coincidence! The Rec. 709 transfer function standardized for 525/59.94 studio video, 625/50 studio video, and HDTV.
Figure 9 The Rec. 709 transfer function of video mimics the lightness sensitivity of vision. The standard is based on a power function with an exponent of 0.45. Theoretically, a pure power function suffices for gamma correction. In a practical system such as a television camera, the slope of the function is limited near zero in order to minimize noise in the dark regions of the picture.
A summary sketch of gamma in video is shown in Figure 10 below. At the camera, luminance (or, in a color system, a set of three tristimulus values) is subjected to the Rec. 709 transfer function – or loosely, gamma correction – whose graph resembles the lightness sensitivity of vision. Video data is stored, processed, recorded, and transmitted in the perceptual domain. The monitor inverts the transform. The main purpose of gamma correction is to code luminance into a perceptually uniform domain, so as to obtain the best perceptual performance from a limited number of bits in each of R’, G’, and B’. (The prime symbols denote the nonlinearity.)
Nonlinear encoding involves applying a transfer function similar to the lightness sensitivity of human vision. Ideally, luminance would first be matrixed, that is, formed as a weighted sum of linear-light (tristimulus) RGB signals. Then, the CIE L* transfer function would be applied, to code the signal into a perceptually uniform domain. At the decoder, the inverse of the L* function would restore luminance, then the inverse matrix would reconstruct RGB. The L* signal would be accompanied by two other signals, to enable the representation of color:
Coding L* with eight bits achieves good image quality. Coding Y directly would require 11 bits or more to achieve similar quality.
使用 8 位编码 L* 可以实现良好的图像质量，直接编码 Y 则需要 11 位或更多才能达到相似的质量。
As I have outlined, the electron gun of a CRT monitor introduces a power function having an exponent of about 2.5:
如我所述，CRT 显示器的电子枪引入了一个指数约为 2.5 的幂函数：
If we were to encode according to the L* function, the decoder would have to invert that function, then impose the inverse of the 2.5-power function of the CRT:
The CRT’s power function is so similar to the inverse of the L* function that we make an engineering compromise: Instead of encoding Y using the L* transfer function, we encode RGB intensities to the inverse of the CRT’s function. This allows us to dispense completely with transfer function circuitry at the display. We must then interchange the order of the matrix and the transfer function at the encoder. Changing the order of operations causes a departure from the Principle of constant luminance. In theory, the encoder would require a 0.4-power function:
This arrangement reproduces physical luminance correctly. However, it has a serious problem: The pictures do not look very good! When viewing a reproduced image, human viewers prefer a reproduction whose contrast ratio has been stretched slightly to a reproduction that is physically correct. The subjective preference depends somewhat upon the viewing environment. In effect, the visual system of the viewer imposes a power function with an exponent of about 1 ⁄ 1.25 :
For television, a power function with an exponent of about 1.25 must be applied to overcome this effect, in order to produce images that are subjectively pleasing. Rather than introducing circuitry at the display to apply this function, we modify the transfer function at encoder. We use an exponent of about 0.5, instead of the physically-correct 0.4:
If you think of encoding in physical terms, you could consider a video image to be encoded such that a 2.0-power function would reproduce physically-correct luminance at the display. (The NTSC video standard is often said to have gamma of 2.2, because the FCC standards described gamma in this way.) However, I think it is more evocative consider the application of the power function at the encoder to impose a rendering intent upon the image data. The encoding assumes the image is to be reproduced in a subjectively-acceptable manner through a physical 2.5-power function at the display.
Though ubiquitous in video, this subjective correction is rarely considered explicitly in computer graphics; belief in the “bits are bits” philosophy suggests to programmers that luminance should be reproduced in the physically-correct manner. However, subjective correction is as necessary in computer graphics as it is in video, and the Rec. 709 transfer function is appropriate for computer graphics. In traditional computer-generated imagery (CGI), as in Figure 8, the subjective correction is typically accomplished by “gamma correction” using a 1 ⁄ 2.2 -power function (instead of 1 ⁄ 2.5 ). This is called “gamma of 2.2”.
The Rec. 709 transfer function is standard for 525/59.94 and 625/60 conventional video, and for HDTV. The Rec. 709 function is based on a power function exponent of 0.45, but the pure power function is modified by the insertion of a linear segment near black. The overall function is very similar to a square root. For details, consult the Gamma chapter in my book. In the diagrams in this section, I use the notation 0.5 as shorthand for the Rec. 709 function. [5]
Rec. 709 appears to strictly define the transfer function at the camera. However, real video cameras have controls that can alter the transfer function. These controls are routinely used by cinematographers and videographers to achieve their artistic intents. Obviously the artistic intention of the cinematographer must be imposed at the camera, not at the display – it ought to be the displays that are standardized, not the cameras! But there is no mechanism to impose standards on displays, so we standardize the reference transfer function at the camera instead. In effect, Rec. 709 is standardized so as to produce acceptable reproduction on a conventional display. Despite the lack of standards, CRT displays are tacitly considered to have similar response.
The engineering of video systems – and, by extension, of desktop computer systems – involves an implicit assumption about the 2.5-power function of the monitor. Alternate display devices, such as LCDs, plasma panels, DMDs, and so on, do not have the 2.5-power function of the CRT. But the most important aspect of image coding is the establishment of a nearly perceptually-uniform image code. In a closed system employing an alternate display technology, you might be tempted to use of a transfer function at encoding that is the inverse of the transfer function at the display. However, if the transfer function of the display was very different than a 2.5-power function, more than 8 bits would be required to code luminance.
More significantly, there is a huge installed base of encoding and decoding equipment that assumes image coding similar or identical to that of video. The installed base includes roughly 1,300,000,000 television receivers, 400,000,000 VCRs, 250,000,000 camcorders, and 300,000,000 desktop computers. These devices are all, in effect, wired to directly reproduce R’G’B’ signals encoded according to Rec. 709. Any proposal for a new encoder transfer function would compromise the interchange of images among these systems.
I have discussed the reproduction of black-and-white images. These concepts extend into the domain of luma and color difference coding, used in video, JPEG, and MPEG. At a SMPTE conference, I discussed the effect of transfer functions in luma and color difference coding. [6]
The block diagrams of pseudocolor, hicolor, and truecolor systems used in desktop computing are sketched in Figures and opposite. These sketches show the hardware pipeline from the framebuffer to the monitor. The interface from application software to the graphics subsystem (and window system) assumes the same processing. Comparable processing is implicit in file formats for pseudocolor and truecolor images. (File formats for hicolor are rare.)
A pseudocolor image is always accompanied by its color lookup table (CLUT). The CLUT may be optimized for the particular image, or it may contain a system palette. Upon display of a pseudocolor image, the graphics subsystem may directly load the colormap that accompanies the image. Alternatively, the graphics subsystem may recode the image according to some other map that is already loaded into the hardware, or according to the system palette native to the application. Recoding of pseudocolor image data may introduce color errors. Pseudocolor image data is always coded in terms of monitor R’G’B’ – that is, pseudocolor image colors are implicitly perceptually coded.
Figure 11 Pseudocolor (8-bit) graphics systems are common in low-end PCs. For each pixel, the framebuffer stores a color index value (typically 8 bits). Each index value is mapped, through a color lookup table (CLUT) that is part of the display hardware, to a triplet of R’G’B’ codes. When a pixel is accessed from the framebuffer, the corresponding triplet is accessed from the CLUT; those values are applied to the digital-to-analog converter (DAC). R’G’B’ codes from the CLUT translate linearly into voltage applied to the monitor, so code values are comparable to video R’G’B’ values – the R’G’B’ values are proportional to displayed intensity raised to the 0.4 power, comparable to video R’G’B’ codes.
A hicolor system has no lookup tables. Image data is coded in terms of monitor R’G’B’: Image data is implicitly perceptually coded (though coarsely quantized).
Figure 12 Hicolor (16-bit) graphics systems store, in the framebuffer, R’G’B’ codes partitioned into three components of five bits each (5-5-5), or partitioned five bits for red, six bits for green, and five bits for blue (5-6-5). In low-end systems, these codes are applied directly to the DACs with no intervening colormap. Because the R’G’B’ codes are translated linearly into monitor voltage, the code values are implicitly proportional to displayed intensity raised to the 0.4 power.
In truecolor, each of the RGB channels is associated with a lookup table (LUT) that applies a transfer function. (Ordinarily, the three tables have identical contents.) Different default lookup tables are in use, for different platforms. Truecolor image files are ordinarily stored without any lookup tables; most truecolor file formats make little or no provision for conveying the transfer function that is expected at display.
Figure 13 Truecolor (24-bit) graphics systems store 8 bits for each of the red, green, and blue components. Truecolor systems usually implement a set of lookup tables (LUTs) between the framebuffer memory and the DACs. The LUTs allow a transfer function to be imposed between the R’G’B’ codes and the DACs: The R’G’B’ values in the framebuffer need not be related to displayed intensity raised to the 0.4 power. Application software or system software can impose arbitrary functions. In order for the application software to provide the same default behavior as low-end pseudocolor and hicolor graphics systems, each LUT is set by default to a ramp corresponding to the identity (or unity) function.
Hicolor and truecolor display hardware can typically be operated in pseudocolor mode. But this mode switch applies to the whole display. If a pseudocolor image is to be displayed in a window of a display that is operating in hicolor or truecolor mode, the graphics subsystem must perform the pseudocolor color lookup operation in software. If the truecolor system is operating with a LUT that is not a ramp, then the RGB codes from the pseudocolor CLUT must be mapped through the inverse of the truecolor LUT prior to being stored in the framebuffer.
If a hicolor or truecolor image is to be presented on a display that is operating in pseudocolor mode, the graphics subsystem must find, for each hicolor or truecolor pixel (RGB triplet), the index of the closest color that is available in the CLUT currently in use. If the CLUT is organized systematically, then this operation can be fairly rapid; if the CLUT is unstructured, then the conversion proceeds slowly. The translation to pseudocolor causes coarse quantization of the image colors. Dithering may be applied, to spread the quantization error over a small area of the image. In any event, colormap quantization generally causes serious degradation of color fidelity.
A generic application program on a PC must assume the lowest common denominator of display capability: It must be prepared to operate without a lookup table. (Even some graphics cards with 24-bit capability have no lookup tables.) Even if a hardware lookup table is present, PC software generally operates as if there is no table. If a LUT is present, it is ordinarily loaded with a ramp so that it has no effect. Image data exchanged among PCs is therefore coded as monitor R’G’B’. Though the situation arose by accident, this is quite comparable to video coding, and is nearly optimal for perception! Image data that originates on (or is intended for display on) a PC carries the implicit assumption that the lookup table contains a ramp, that is, that the image data is represented in gamma-corrected monitor R’G’B’. So Figure 10 applies to video and to PCs: the coding is comparable.
在 PC 上的通用应用程序必须假设能够兼容最低的显示能力：它必须准备在没有查找表的情况下运行。（即使某些具有 24 位功能的显卡也没有查找表。）即使存在硬件查找表，PC 软件通常也假设不存在查找表。如果存在查找表，它通常被加载为斜坡函数，因此没有效果。所以，在 PC 之间交换的图像数据以显示器的 R’G’B’ 编码。尽管这种情况偶然发生，但这与视频编码非常相似，并且几乎是最佳的感知方式！在 PC 上生成（或用于在 PC 上显示）的图像数据包含一个隐含假设，即查找表包含斜坡函数，即图像数据以伽马校正后的显示器 R’G’B’ 表示。因此，图 10 适用于视频和 PC：编码是相似的。
Macintosh gamma 苹果电脑的伽马
Contrary to popular belief, Macintosh computers use monitors that have the same physics as monitors used in video systems and other brands of computers. Though it is nowhere documented in the 27 volumes of the Inside Macintosh series of books, the QuickDraw graphics subsystem loads an unusual transfer function into the lookup tables of a Mac. The default lookup table (in hexadecimal code) is this: [7]
与流行的看法相反，Macintosh 计算机使用的显示器与视频系统，和其他品牌计算机使用的显示器具有相同的物理特性。尽管在《Inside Macintosh》系列的 27 本书目中没有任何文档记录，但 QuickDraw 图形子系统会在 Mac 的查找表中加载一个不寻常的转换函数。默认的查找表（以十六进制码值表示）是这样的：
Figure 14 Default Macintosh LUT
图 14：默认的 Macintosh 查找表
This table contains a pure power function with an exponent of 1 ⁄ 1.45 . Image data that originates on – or is intended for display on a Macintosh computer carries the implicit assumption that the lookup table contains this function. This default lookup table, in combination with a conventional monitor, causes the R, G, and B values presented to QuickDraw to represent the 1 ⁄ 1.8 -power of luminance.
这个查找表包含一个指数为 1/1.45 的纯幂函数。在 Macintosh 计算机上生成的（或打算显示的）图像数据，默认假设查找表包含这个函数。这个默认查找表与传统显示器特性结合后，使得提供给 QuickDraw 的 R、G 和 B 值表示为光亮度的 1/1.8 次幂。
Although Apple has historically failed to publish any meaningful documentation of gamma, a Macintosh is widely considered to have a default gamma of 1.8. This de facto nomenclature was established by the Gamma control panel, by Knoll Software, which was distributed with Adobe Photoshop up to and including version 4.
虽然 Apple 在历史上从未发布任何关于伽马的详细文档，但 Macintosh 的默认伽马值通常被认为是 1.8。这一事实上的标准是由 Knoll Software 的 Gamma 控制面板确立的，该面板随 Adobe Photoshop 分发，包括第 4 版在内。
Figure 15 below summarizes the gamma situation for Macintosh.
下图 15 总结了 Macintosh 电脑的伽马情况。
Figure 15 Gamma in Macintosh
图 15：苹果 Macintosh 电脑的伽马
In the Gamma chapter of my book, I explain the dot gain phenomenon of offset printing. Offset printing uses code values proportional to the 1.8-power of reflectance. But QuickDraw RGB codes are related to luminance by the 1.8 power! Though the situation arose by accident, QuickDraw RGB coding is well suited to offset printing, and it is ubiquitous in desktop publishing and prepress. (QuickDraw coding is also widely used in multimedia, and on the World-wide Web, though in these applications it is not as suitable as coding according to Rec. 709.) [8]
Figure 16 above collects the three gamma sketches already presented (video, computer-generated imagery, and Macintosh), and adds a fourth sketch, for Silicon Graphics (SGI). Given the diverse transfer functions, it is no surprise that it is difficult to exchange image data.I have indicated in bold type the numerical quantity that is referred to as gamma in each of these four domains: You can see that the gamma number is applied in four different places! So even if you know the gamma value, it is difficult to determine where it is applied! As I have mentioned, a Macintosh is considered to have a default gamma of 1.8. An SGI computer has a default gamma of 1.7. But Figure 16 shows that these two numbers are not comparable!
Figure 16 Gamma in video, PC, computer graphics, SGI, and Mac
图 16：视频、PC、计算机图形、SGI、Mac 的伽马
The graphics subsystems of most computers allow the lookup table to be changed. On a Mac, the Gamma control panel accomplishes this. When a gamma value of g is specified, the control panel loads the Mac lookup table with a power function whose exponent is g ⁄ 2.61 . When a gamma value of g is specified to an SGI computer, system software loads the SGI lookup table with a power function whose exponent is 1 ⁄ g . The convention differs from that on a Mac. To program an SGI computer to behave like a Mac, you must set SGI gamma to 1.45.
大多数计算机的图形子系统允许更改查找表。在 Mac 上，Gamma 控制面板完成此操作。当指定伽马值 g 时，控制面板加载一个指数为 g/2.61 的幂函数到 Mac 的查找表中。当指定伽马值 g 到 SGI 计算机时，系统软件加载一个指数为 1/g 的幂函数到 SGI 的查找表中。这一约定与 Mac 不同。要使 SGI 计算机表现得像 Mac，必须将 SGI 伽马设置为 1.45。
Image data in the framebuffer is not usually changed upon a change of the lookup table. Any time you jam a particular value of gamma into the back end of the graphics subsystem, you override assumptions that may have been made about the color interpretation of image data. When you change gamma, the colors of displayed objects (icons, menus, and windows) and the colors of displayed images, will change!
To exchange images using computer graphics standards requires knowledge of the transfer function. Standards such as PHIGS and CGM stem from computer-generated imagery (CGI), where linear-light (tristimulus) coding is the norm, and in PHIGS and CGM it is implicit that RGB data is coded in linear-light (tristimulus).
JPEG and other lossy image compression algorithms depend on discarding information that won’t be perceived. It is vital that the data presented to a JPEG compressor be coded in a perceptually-uniform manner, so that the discarded information has minimal perceptual impact. In practice, JPEG works well only on nonlinearly-coded (gamma-corrected) image data.
But nowhere in the PHIGS, CGM, or JPEG standards is gamma or transfer function explicitly mentioned, and nowhere in the data streams or image file formats for PHIGS, CGM, or JPEG, is the transfer function conveyed! The user must handle the transfer function or face poor image quality. If image data is transferred between these systems without regard for the transfer function, then the pictures will have terrible quality. Figure 17 below summarizes the situation: RGB codes [128, 128, 128] produce completely different intensities at the face of the screen in PHIGS (or CGM) and JPEG. But the standards themselves provide absolutely no information concerning this issue.
Figure 17 Gamma in PHIGS, CGM, and JPEG. In the PHIGS and CGM standards there is no mention of gamma or transfer function, but it is implicit that image data is coded as linear-light tristimulus values. In the JPEG standard there is no mention of gamma or transfer function, but it is implicit that image data is gamma corrected.
Many other computer graphics standards ignore or discount transfer functions. Figure 18 above shows a screenshot of the Apple Macintosh Color Picker, which implements the textbook HSL representation of color. This presentation implies that all of the colors shown share the same lightness value, but clearly the disk is not uniformly shaded. The HSL representation has no objective basis in color science.
许多其他计算机图形标准也忽略或轻视转换函数。下图 18 显示了 Apple Macintosh 颜色选择器的截图，它实现了教材中的 HSL 颜色表示。这种表示暗示所有显示的颜色具有相同的明度值，但显然圆盘并没有均匀着色。HSL 表示在颜色科学中没有客观基础。
Figure 18 Lightness in HSL. Lightness, in its CIE definition, is a perceptual quantity. However, in the textbook HSL color representation used in computer graphics – here exemplified by Apple’s Macintosh Color Picker – no account is taken of transfer function. Apple implies that all of the shades in the disk have the same lightness of 50%. Does the disk appear uniformly shaded to you?
图 18：HSL 中的明度。根据 CIE 的定义，明度是一个感知量。然而，在计算机图形中使用的教材 HSL 颜色表示中——这里以 Apple 的 Macintosh 颜色选择器为例——没有考虑转换函数。Apple 暗示圆盘中的所有阴影具有相同的 50% 明度。这个圆盘看起来是均匀着色的吗？
World-wide web 万维网
The World-wide web uses GIF and JPEG file formats to convey images. (Other file formats are in use, but none of these are widely deployed.)
A GIF file represents an image in pseudocolor form. A web browser operating on a pseudocolor display does not attempt to reconcile the potentially conflicting CLUTs found among the several (or several dozen) GIF images that might share a window on the user’s monitor. Instead, a browser typically recodes every pseudocolor image into a browser palette comprising a 6×6×6 colorcube of monitor R’G’B’ codes. The browser palette comprises 216 colors. (To display GIF images on a hicolor or truecolor system, the browser’s graphics subsystem uses each file’s CLUT to translate each image to R’G’B’.)
JPEG image coding is based on truecolor. The JFIF specification is the de facto standard for JPEG file interchange. JFIF is unclear concerning the handling of transfer function. In practice, an image is encoded into JPEG using the encoding transfer function that is in effect for the platform that it is encoded on. A decoded JPEG image is displayed using the transfer function in effect on the platform upon which it is decoded. Figure 19 overleaf sketches the gamma situation for JPEG on the web. It’s chaos! Image data is exchanged among platforms without regard for the transfer function that will be applied upon display. The same file displays differently on different platforms!
Figure 19 Gamma on the web. Image data is exchanged on the web without regard for the transfer function that will be applied at display. Consequently, the same image is displayed differently on different platforms. Here, the image originates on an SGI computer. In practice, the dominant platforms are PC and Mac.
图 19：网络上的伽马。图像数据在网络上交换时不考虑显示时将应用的转换函数，因此，同一图像在不同平台上的显示效果不同。这里的图像源自 SGI 计算机。实际上，主导平台是 PC 和 Mac。
In Figure 19 above, I show an image originating on an SGI computer. You can see that it is decoded in four different ways; in particular, it is reproduced incorrectly on PC and Mac, the dominant viewing platforms. If a JPEG image originates on a PC, it displays incorrectly on a Mac; if a JPEG image originates on a Mac, it displays incorrectly on a PC. The relatively low penetration of Macintosh computers in the marketplace might suggest that image origination should be optimized for PCs: let the Mac browsers do what they will. But the tools for image preparation and web page creation are much more capable on Macs than on PCs. A large fraction of web images are prepared on Macs, so the implicit Mac transfer function is important. [9]
在上图 19 中展示了一个源自 SGI 计算机的图像。你可以看到它在四种不同的方式下解码，尤其是在主流查看平台 PC 和 Mac 上，它显示不正确。如果 JPEG 图像源自 PC，它在 Mac 上显示不正确；如果 JPEG 图像源自 Mac，它在 PC 上显示不正确。由于市场上 Macintosh 计算机相对低的渗透率，可能会认为图像生成应该主要针对 PC 进行优化，而让 Mac 浏览器自行调整。但用于图像准备和网页创建的工具在 Mac 上更为强大。因此，很多网页图像是在 Mac 上准备的，隐含的 Mac 转换函数非常重要。
The second row of the sketch shows the CGI situation. In practice, images are rarely displayed through a LUT configured for CGI. But the VRML language stems from CGI, and shading in synthetic computer graphics has historically been performed in the linear-light domain. (Shading in nonlinear domains is poorly understood.) VRML is best considered to originate in the computer graphics row of the sketch.
Work is underway to implement facilities in graphics systems to allow device-independent specification of color. Users and applications will be able to specify colors, based on the CIE standards, without concern for gamma correction. Color image files will be tagged with their transfer functions (along with other color parameters). When this transition is complete, it will be much easier to obtain color matching across different graphics libraries and different hardware.
However, these developments will not render gamma irrelevant. Proper use of transfer functions will remain necessary in order to code images in a perceptual manner, so as to achieve maximum performance from a reasonable number of bits per component – 8, say! Also, it will take a long time for this technology to be deployed.
In the meantime, if you are a programmer, you can take the following steps:
与此同时，如果你是程序员，可以采取以下步骤：
Use gamma-corrected R’G’B’ representations whenever you can. An image coded with gamma correction has good perceptual uniformity, resulting in an image with much higher quality than one coded as 8-bit luminance (or tristimulus) values.
When you exchange images either in truecolor or pseudocolor form, code R’G’B’ color values using the Rec. 709 transfer function.
在以真彩色或伪彩色形式交换图像时，使用 Rec. 709 转换函数编码 R’G’B’ 颜色值。
In the absence of reliable information about your monitor, display pictures assuming a monitor gamma value of 2.5.
在缺乏可靠的显示器信息时，假设显示器的伽马值为 2.5 进行显示。
If you are a user, you can take these steps:
如果你是用户，可以采取以下步骤：
Establish good viewing conditions. If you are using a CRT display, you will get better image quality if your overall ambient illumination is reduced.
建立良好的观看条件。如果你使用的是 CRT 显示器，降低整体环境照明将提高图像质量。
Ensure that your monitor’s BLACK LEVEL (or BRIGHTNESS) control is set to correctly reproduce black elements on the screen. Consult my note “BRIGHTNESS” and “CONTRAST” controls, available on the Internet.
Demand, from your hardware and software developers and vendors, that they document how they handle transfer functions.
要求你的硬件和软件开发人员及供应商记录他们是如何处理转换函数的。
1.See Olson, Thor, “Behind Gamma’s Disguise,” in SMPTE Journal, v. 104, p. 452 (June 1995). ↩
2.Berns, Roy S., Ricardo J. Motta, and M.E. Gorzynski, “CRT Colorimetry: Part 1, Theory and Practice; Part 2, Metrology,” in Color Research and Application, v. 18, 299–325 (1993). ↩
3.Publication CIE No 15.2, Colorimetry, Second Edition. (Vienna: Central Bureau of the Commission Internationale de L’Éclairage, 1986) ↩
4.Stokes, Mike, Mark D. Fairchild, and Roy S. Berns, “Precision requirements for digital color reproduction,” in ACM Transactions on Graphics, v. 11, n. 4 (Oct. 1992), 406–422. ↩
5.Poynton, Charles, A Technical Introduction to Digital Video (New York: Wiley, 1996). ↩
6.Poynton, Charles, “Luminance, luma, and the migration to DTV,” presented at 32nd SMPTE Advanced Motion Imaging Conference, Toronto (Feb. 6, 1998). ↩
8.Poynton, Charles, A Technical Introduction to Digital Video (New York: Wiley, 1996). ↩
9.Gille, J.L., J.O. Larimer, and J. Luszcz, “Error diffusion using the ‘web-safe’ colors: how good is it across platforms?”, in Rogowitz, B.E. and T.N. Pappas (eds.), Human Vision and Electronic Imaging III, Proceedings of SPIE, Volume 3299, 368–375 (Bellingham, Wash.: SPIE, 1998). ↩
出处：《Handbook of Research on the Global Impacts and Roles of Immersive Media》 标题：Immersive Storytelling: Leveraging the Benefits and Avoiding the Pitfalls of Immersive Media in Domes 作者：Michael Daut, Independent Researcher, USA 翻译：Horace Lu
（注：键盘快捷键“w”或左侧菜单右上角按钮，可切换文章列表视图与大纲视图）
[TOC]
摘要 ABSTRACT
This chapter compares and contrasts the development of traditional cinema and fulldome cinema, describing the way their origins shaped not only their current success and potential as unique cinematic mediums, but also how their cinematic languages developed. There is a vastly different approach to storytelling that filmmakers must understand when creating shows for immersive digital dome theaters versus the approach they would take to tell stories in a traditional film. This chapter identifies key differences between cinema and fulldome and provides a primer for immersive storytelling on the dome from understanding the technology to understanding how most effectively to use the strengths of fulldome while avoiding its weaknesses. Ultimately, this discussion is designed to help creative artists become more effective immersive filmmakers for the fulldome canvas.
Modern society is experiencing an explosion of immersive media in a nearly overwhelming number of forms: Virtual Reality (VR) that requires a headset that feeds 360º imagery in the user’s eyes and pours immersive audio into their ears; Augmented Reality (AR) that creates visual and auditory overlays on top of reality using a smart phone or a semi-transparent pair of glasses; Mixed Reality (MR), that uses a combination of VR and AR to create new and unexpected experiences through a headset that can change from fully transparent to fully opaque based on the content creator’s design. Then there are hybrid forms of immersive media that blend theatrical stagecraft with a VR system that allows free roaming through physical spaces with walls people can see virtually and touch in reality, props they can use, and other tactile sensations that powerfully blur the lines between virtual and reality. These are just some examples of immersive media that involve some sort of device that the audience must either use or wear, and more times than not, these “vehicles to immersion” create a sense of isolation, not a shared community experience.
在当今社会，沉浸式媒体正以一种几乎令人应接不暇的形式迅速发展：虚拟现实（VR）需要用户戴上头显，360 度的影像直接呈现在眼前，沉浸式的音频环绕耳边；增强现实（AR）通过智能手机或半透明眼镜，在现实世界之上叠加视觉和听觉效果；混合现实（MR）则结合了 VR 和 AR 的特点，通过可根据内容创作者的设计从完全透明变为完全不透明的头显，创造出新颖而出人意料的体验。此外，还有将戏剧舞台技术与 VR 系统结合的混合型沉浸媒体，它允许人们在有实体墙壁的物理空间中自由漫游，使用可在现实中触摸到的道具，以及其他强烈的触觉感受，极大地模糊了虚拟与现实之间的界限。这些仅仅是沉浸式媒体的一部分例子，它们通常需要观众使用或佩戴某种设备，而更多时候，这些“沉浸之舟”带来的是孤立感，而非共享的集体经验。
On a more basic level there is 3D stereo technology that exists in cinema, VR, home theater, video games, lenticular stereo printing, giant screen theaters, and even giant 3D dome theaters to add visual depth to the experiences. In a completely different type of experience, interactive media platforms like Twitch add to the viewer’s sense of agency and therefore immersion.
在更为基础的层面上，3D 立体技术已经存在于电影院、虚拟现实（VR）、家庭影院、电子游戏、立体印刷、巨幕影院，甚至是巨型 3D 球幕影院等媒介中，为观众带来了视觉深度上的增强体验。像 Twitch 这样的互动媒体平台则提供了另一种完全不同的体验，增强了观众的主体感（agency）和沉浸感（immersion）。
These “new media” experiences have brought with them new ways of telling stories and a new type of cinematic and aesthetic language that creatives and consumers alike are still trying to understand and unravel. New media storytellers are experimenting with new ways of immersive expression and developing and inventing a new lexicon of techniques and understanding how to speak this immersive visual language. It is an exciting time as creatives are blazing a trail through this largely undiscovered country. Exploring the art of immersive storytelling opens a deep well that branches in nearly infinite directions that would overwhelm this chapter and spill over into a series of books.
This chapter focuses on a specific type of immersive medium: digital fulldome theaters (Figure 1). From their origins as planetarium spaces to their continuing growth into VR Theaters of the future, this exciting medium has developed its own cinematic language that is part traditional cinema, part live theater, and a lot of something magical that when leveraged effectively can transport audiences as a small community into shared virtual experiences. Technological advancements and system features still impact digital domes as much as the format’s differences from traditional cinema. How has cinematic language developed in traditional cinema, and how has it formed in digital immersive domes? How can these languages be the same? How must they be different? Are immersive digital fulldome theaters effective spaces for storytelling, or are these spaces best used for documentary-style programs and purely educational experiences? These questions are only the jumping-off points for this fascinating exploration.
Modern cinema was born with the invention of the motion picture camera, and as often happens, many pieces of this technology were being developed simultaneously and independently by a number of inventors across the world. It is therefore difficult to pinpoint exactly who invented the motion picture camera, although most historians attribute this honor to American innovator, Thomas Edison, who incidentally would have willingly accepted this attribution. The truth of the matter is more complicated, of course, with Edison’s Kinetograph, motion picture camera, built upon the work of early pioneers, Francis Ronalds, Wordsworth Donisthorpe, Louis Le Prince, William Friese-Greene, and William Kennedy Laurie Dickson. Even the exact year the world-changing camera came into being is up for debate. For further background on the story behind the creation of the motion picture camera, explore the many resources listed in the Additional Reading section at the end of this chapter.
Regardless of its specific origins or authorship, in 1891 or 1892, the motion picture camera ushered in the age of cinema. Cinema exhibition started with Edison’s Kinetoscope (Figure 2): a mechanical device in which one person at a time would look through an eyepiece or “peephole” at the top to view the images printed sequentially on a strip of film. Sprocket holes along the film’s edges allowed the film to be pulled through the device across an illuminated image plate below the peephole. A high-speed shutter hid the transition between frames, completing the illusion of movement. Later the Lumi è re brothers would transform the Kinetoscope into the Cinématographe, the first motion picture projector, allowing an entire audience to view the film at once (Library of Congress, n. d.).
Figure 2: A person watching a movie with Edison’s Kinetoscope Source: Public domain image from Wikimedia Commons
Figure 3. The interior of Edison’s Kinetoscope showing the film path Source: Public domain image from Wikimedia Commons
It took over two decades for filmmakers to truly understand and exploit its visual language. D.W. Griffith was one of the early masters of visual cinematic storytelling, using the close-up and careful editing to build the experience shot by shot. Legendary pioneering filmmakers such as Buster Keaton, Charlie Chaplin, Cecil B. De Mille, and others further contributed their legerdemain to advance the art. In step with (or more accurately, many steps behind) developing creative and aesthetic techniques, the technology itself advanced, sometimes at a snail’s pace; other times blindingly fast, creating an interdependency that exists to this day. Filmmakers developed their stories within the confines of the technology while pushing diligently against technology’s limits. At the same time, the inventors both responded to the creative needs and leveraged new, even groundbreaking developments to reinvent the process of motion picture image capture and exhibition (and all the steps in between) over and over again with nearly relentless fervor. In the same way, filmmakers would leverage these new innovations to further elevate visual storytelling and define new ways to create movies in the ever-changing technological landscape. This interdependent growth cycle between art and technology is not unique to cinema. The same patterns can be seen in computer technology, science, engineering, music, even accounting and finance. Technology creates opportunity; users drive technology; technology advances creating new opportunities; users find new applications and drive technology further; technology advances again, and the cycle goes on indefinitely. The cycle itself is not as crucial to this discussion as is the interdependence of art and technology. They cannot be separated, and they enable one another. Just as oil paints as a medium influence and affect the outcome of the art on the canvas, so the technology of cinema affects those who paint with its cinematic language. The creation of the art must leverage the available tools and the strengths and limitations of the medium to harness its full potential. This required deep understanding and experience on the part of the artist.
电影人花了二十多年的时间才真正理解并开始应用电影的视觉语言。D.W. 格里菲斯是视觉电影叙事的早期大师之一，他运用特写镜头和精心的剪辑构建逐个镜头的体验。巴斯特·基顿、查理·卓别林、塞西尔· B ·德米尔等传奇的先驱电影人进一步贡献了他们的巧妙手法，推动了这门艺术的发展。随着创意和审美技巧的发展，技术也在不断进步，但这种进步有时非常缓慢，有时则是突飞猛进。正是这种时而缓慢时而迅速的发展，构建了创意与技术之间持续至今的相互依存关系。电影人在技术的局限下拓展他们的故事，同时孜孜不倦地挑战技术的极限。与此同时，发明家们一方面响应创意需求，另一方面也带着不懈的热情，利用新的、甚至是开创性的进展，一次次地重新发明活动影像的拍摄和放映流程（以及两者之间的所有步骤）。同样地，电影人也会利用这些创新来进一步提升视觉叙事，并在不断变化的技术环境中定义制作电影的新方法。艺术与技术的这种相互依赖的增长周期并不是电影独有的，在计算机技术、科学、工程、音乐，甚至会计、金融等领域，都可以看到同样的模式。“技术创造机会；用户推动技术；技术进步创造新机会；用户发现新应用并进一步推动技术；技术再次进步”，这个周期无限循环。与这种周期相比，艺术与技术的相互依赖对于本讨论更为关键，它们不可分割、相互促进。正如油画颜料作为绘画的媒介会影响画布上艺术作品的表现力一样，电影技术同样影响着那些运用电影语言进行创作的艺术家。艺术创作必须利用可用的工具及媒介的优势和局限来发挥其全部潜力，这需要艺术家对此有深刻的理解和丰富的经验。
数字球幕系统的发展 The Development of Immersive Digital Dome Systems
Just as motion pictures, immersive dome systems began with analog technology. In 1923 the Zeiss Corporation in Jena, Germany developed a revolutionary optical mechanical projection device, a metal sphere studded with lenses and a cylindrical appendage that extended underneath, affectionately called, The Wonder of Jena, that could realistically reproduce the starry night sky. This device required a new physical structure and projection surface that approximated the apparent shape of the sky overhead. A large hemispherical dome, 360 ° around by 180 ° high became the obvious choice (Lambert, 2012).
The world’s first planetarium opened at the Deutsche’s Museum in Munich, Germany in 1925 with the first Carl Zeiss optical mechanical star projector. The projector used light inside of a mechanism that could project points of light that replicated the starry sky as seen from Earth, enabling them to move across the sky as night transitioned to day and back again into night. It could even adjust its orientation to show the accurate night sky from any location on Earth. This optical mechanical projector also featured planet projectors that through precise mathematical calculations could mirror the forward and retrograde movement of planets through the night sky. This technology caught worldwide attention, inspiring a generation of astronomers and science educators. The first planetarium in the US, the Adler Planetarium in Chicago, opened its planetarium in 1930, with other, now legendary planetariums, like the American Museum of Natural History in New York, and Griffith Observatory in Los Angeles opening shortly thereafter (Marche, 2005).
It goes without saying that Carl Zeiss and other companies that followed, designed and built domes with the express purpose of showing audiences the wonders of the night sky. These theaters provided astronomy education spaces where people could learn about and identify constellations, planets, and seasonal changes in the sky. The original theater and seating design that supported this purpose consisted of a hemispherical dome screen hung directly above the audience with its base completely parallel to the ground (with no tilt forward or backward in any direction). The audience sat in concentric rows with their backs to the outside walls, each guest facing into the center of the theater where the optical mechanical projector resided. This orientation allowed guests to view the night sky in all directions simultaneously while highlighting the magnificent star projector as the centerpiece to the experience (Figure 4).
Figure 4. Inside a concentric planetarium with a non-tilted 180º dome and star projector Source: Public domain image from Wikimedia Commons
The modern planetarium was born (incidentally, only about 40 years after the birth of the motion picture camera). This new medium that was purpose-built for teaching astronomy would embark on its own wild ride and hard-fought path to be a legitimate cinematic and immersive medium in its own right, and the battle has only begun.
Even with the star projector’s exquisite ability to recreate the stars and planets of the night sky, the instrument could only depict stellar observation from the surface of earth. The audience could never take off into space and fly through the stars. Because star projectors were large mechanical devices, they could only move the night sky at relatively slow speeds to simulate Earth’s rotation and the eventual sunrise.
In the early 1970’s, Ivan Dryer created Laserium, a visual music experience for planetarium domes. Ivan and his team created a custom laser projector that could be preprogrammed and choreographed to a musical soundtrack and performed live across the dome’s interior in front of an audience. Laserium stepped into history at the Griffith Observatory in Los Angeles on November 19, 1973. For the first time pure entertainment shows entered planetarium domes, and the impact of Laserium continues to inspire producers and inspire new creative uses for the dome’s hemispherical canvas (Ehrman, 2002).
Then in the early 1980’s something revolutionary (and at the time somewhat primitive by today’s standards) happened. Evans & Sutherland (E&S) a company based in Salt Lake City, Utah largely known for their pioneering work in computer graphics, introduced a digital star projector called Digistar, and the planetarium changed forever, but certainly not overnight. In fact, the planetarium is still in the process of changing to this day. Digistar introduced two significant groundbreaking capabilities: 1) the ability to lift off of Earth and fly through the stars (and view them from essentially any point in the known universe and at any point in history); 2) the ability to create vector graphics allowing the display of any sort of wireframe computer graphics imaginable. These digital tools expanded the types of experiences that were possible inside planetarium domes.
Because of Laserium and Digistar’s paradigm shifts, the dome theater started to become more than just a planetarium, but its metamorphosis would be slow: very, very slow.
The 1990’s introduced yet another giant leap forward: fulldome video (Figure 5). The concept was straightforward and ingenious. Take a number of video projectors—initially six—install them in a circular arrangement at the bottom edge of the dome screen and project imagery from these projectors across the dome to create one fulldome video display. These projected images would overlap each other, creating regions where the images could be blended together. Precise, painstaking alignment of the blended images would create the illusion of a single fulldome video. A cluster of PCs or a real-time computer image generator (essentially a powerful custom computer built for flight simulation) processed a sequence of circular fisheye images into the system, sliced into tiles that would synchronize and feed each individual projector, allowing video to display across the fulldome canvas.
Figure 5. Inside a digital fulldome theater Source: Adam Kozak distributed under a CC-BY 2.0 license
Once fulldome video existed, dome productions could tell more immersive and sophisticated visual stories, but the fact that domes were nearly exclusively found in planetariums, most fulldome content stayed within the astronomy genre. This is a critical reason that it has been (and continues to be) difficult for fulldome films to break free from their astronomical origins and become more mainstream. Theater design shifted from domes that were parallel to the ground with omnidirectional seating to domes that tilted forward in front of the audience with unidirectional seating that faced the front of the dome tilted down anywhere from 10º to 30º. This has produced a daunting inconsistency in theater layout, dome tilts, projector resolution, placement, quality, and seating configurations, which presents significant challenges for producers who hope that their content will look equally good in all venues.
Today’s advancements have dramatically improved image quality in the theaters. Auto-alignment and auto-blending technologies have allowed even more projectors to be tiled together across the screen and look visually seamless, as if projected from a single source. The most advanced domes use laser video projection and enough 4K projectors to create a fulldome canvas that is over 8192 x 8192 pixels (over 64,000,000 pixels). At the time of writing, the most compelling new technology to be introduced is a self-illuminating dome constructed of LED tiles, eliminating the need for video projection.
如今，技术的进步已经显著提高了球幕影院中的影像质量。自动对齐和混合技术可以将更多台投影仪的画面拼接在球幕上，达到视觉上的无缝衔接，就像从单一源投射出来的一样。最先进的球幕采用激光视频投影技术，将足够多的 4K 投影仪组合起来，创造一个超过 8192 x 8192 像素（总计 64,000,000 像素以上）的全景球幕画布。在撰写本文时，最引人注目的新技术是由 LED 模块构成的自发光球幕，这种技术使得投影机不再是必需品。
Fulldome’s planetarium origins and the fact that almost its entire network of existing theaters exists within planetariums and science centers have largely prevented content to break free from a science education focus to general entertainment. Planetariums’ general lack of marketing budget and lack of budget in general often prevents them from being able to afford higher quality shows. The low revenue stream from theaters that license fulldome shows has kept production budgets low, maybe 1% or less of the typical budget spent on a Hollywood film.
Still, the powerful immersive nature of the medium is propelling it forward into new types of content and into new and unexpected venues including themed entertainment, pop up experiences, concerts, theatrical performances, and touring shows.
As mentioned in the introduction VR, AR, MR and most other immersive technologies require some sort of apparatus and some degree of individualized isolation for the experiences to work. Not so in fulldome. This exciting medium has the true potential to deliver on the promise of VR while remaining a communal, shared experience. Fulldome is essentially the platform for the VR Theater of today and the future. Fulldome also has the potential to attract audiences in a way that traditional cinema is struggling to maintain.
Because of its uniqueness and its ability to exceed the power of VR, it’s important to understand the distinguishing characteristics of fulldome, its potential, and its cinematic language. The rest of this chapter is designed as a primer, of sorts for the medium and a guide to harnessing its abilities to create a powerful new form of storytelling.
The fulldome medium itself stands at the intersection of theater and cinema, yet it is neither both nor one or the other. It is its own blended experience that can be leveraged to produce a powerful sense of presence within an immersive theater. Also, whereas VR is an individual, isolating immersive experience, akin to Edison’s Kinetoscope, fulldome is a shared audience experience more like the Lumi è re brothers’ Cinématographe motion picture projector.
Unlike traditional cinema, where the audience is looking at a screen, digital fulldome places the audience inside of the screen. Cinema uses a framed window in front of the audience to slowly invite them into the experience through emotional connection and the willing suspension of disbelief. Fulldome cinema surrounds the audience in such a way that they start inside the experience, even before the audience’s engagement can draw them in. This instant or “forced immersion” can be difficult to overcome, especially if the audience feels overwhelmed and pushes back from this intrusion. It’s the cinematic version of coming on too strong romantically. It can be uncomfortable and off-putting. More subtle techniques at the start of a fulldome film/immersive experience should be employed to create an emotional connection or conversely, a fascinating spectacle could be created that entices the audience to want to see more. A live presenter to warm up the audience or even participate in the dome show can function as the audience surrogate in the immersive presentation and bridge this gap even further. Once the audience engages in the experience, the feeling of deep immersion and sensation of simulated reality can be extraordinarily impactful.
The visuals that surround the audience create a strong sense of presence, simulating the sensation that the audience actually exists in the place depicted around them on the dome. This is a powerful illusion that is analogous to subjective camera in traditional cinema, where the audience is seeing through the camera as if they are participating in the scene. This effect is more powerful in immersive fulldome, since there is no perceived or implied camera. The virtual world simply surrounds the audience and transports them into the scene’s setting, much like the legendary Holodeck from Star Trek: The Next Generation (1987).
Cinematically there are two major types of scenes in fulldome: subjective scenes and objective scenes. Subjective scenes (designed to make the audience feel as if the experience is happening to them personally) inherently play in real time, since that is the only way humans can move through time and space. Most transitions should not be used in these sequences, since there are no transitions or gaps in time or space when moving from one place to another in reality. When scene changes are necessary within subjective sequences, dips to black, or a time/space travel device, like a visual time warp, launch to light speed, or some similar visual effect, can preserve the audience’s sense of being a participant in the scene. In the early days of digital fulldome, essentially all scenes were subjective scenes. Producers were concerned about breaking the audience’s sense of presence in time and space. This greatly limited the types of storytelling that could be done and eliminated the possibility for editing and adjustments to the show’s pacing.
全景球幕电影主要分为两类场景：主观场景（subjective scenes）和客观场景（objective scenes）。主观场景旨在让观众感受到事件仿佛发生在自己身上，因而通常以实时发生的形式呈现。在这种场景下，应尽量避免使用转场效果，因为人在现实时空中的移动是连续无缝的。当需要变换场景时，可以使用淡出至黑场的效果，或采用时空旅行的视效，如视觉时间扭曲（visual time warp）或光速飞跃（launch to light speed）等，这有助于保持观众的参与感。在数字全景球幕的诞生初期，几乎所有场景都采用了主观视角，制作团队担心其他拍摄方式可能会破坏观众对时空连续性的感知。这种做法极大地限制了故事类型的多样性，并且排除了通过后期剪辑来调整影片节奏的可能性。
Experiments with editing, specifically cutting, revealed that if properly integrated into a sequence, cuts could produce objective scenes in which the audience felt they were in the experience, but not directly inside the scene. Looking at the main action from a physical and emotional distance, rather that always feeling like they were directly inside of the action. Objective scenes allow the audience members to become close bystanders rather than feeling like the actions in the scene are happening directly to them.
Alternating between subjective and objective scenes allows emotional ebbs and flows in the structure of a show, so that producers can employ subjective techniques to “grab” or “fully immerse” the audience and objective techniques to offer the audience a safe distance from the impact of the scenes.
Finding this balance is one of the most important keys to immersive storytelling on the dome. Misunderstanding and misusing these techniques are the quickest way to break the audience’s willing suspension of disbelief and pull them out of the experience.
Ben Shedd called fulldome a “frameless medium,” which it is to a large degree; however, the audience is almost always looking up to some degree, even in a severely tilted (30º forward tilt) dome. The audience can also see below the edge of the dome and see the edges of the dome to either side of them when facing forward in a unidirectional dome (or even in a non-titled dome with omnidirectional seating). In most fulldome theaters, the audience is aware of at least the lower edge of the frame.
Domes also have the inherent challenge of needing lots and lots of image resolution, since the dome surrounds the audience on all sides, and half of the pixels are behind the audience. This causes the individual pixels themselves to be closer to the audience and larger than they would be if they were all out in front of them. Imagine an HDTV folded over the heads of each audience member. The pixels would appear too large and too close. For optimum resolution, more and smaller pixels are necessary. VR has a similar resolution problem, since even a “retina display” that is too close to the viewer appears to have large pixels and low resolution. This is why many modern fulldome theaters and shows have at least 6K × 6K resolution or even 8K × 8K resolution. The standard master frame for digital fulldome is a circular quidistant azimuthal fisheye with a 1: 1 (or square) aspect ratio. The circular fisheye frame is called a dome original or dome master (Figure 6). Dome master hemispherical sequences are typically produced at 4K × 4K (4096 x 4096 resolution) up to 8K × 8K (8192 x 8192 resolution) for each frame. Fully spherical scenes can also be mastered in equirectangular format (the standard frame format for VR production) at 8192 x 4096 for 4K spherical content or 16384 x 8192 for 8K spherical content.
Figure 6. The dome master or fisheye frame of digital fulldome Source: Photo by Diego Delso, distributed under CC-BY-SA 4.0
Furthermore, standard 24 fps cinema frame rate looks “steppy” and appears to judder in the dome. Higher frame rate (HFR) capability smooths out the experience, especially for digital astronomy presentations. For pre-rendered shows, the standard fulldome frame rate is 30 fps, and for real time graphics, the standard frame rate is 60 fps. Higher frame rates up to 120 fps and higher are also possible 3D stereo fulldome is also possible, requiring additional technology in the theater to play back 3D and a complete dome original frame sequence of left eye frames and right eye frames at a rate of 60 fps per eye.
此外，标准的 24 fps 电影帧率，在球幕上看起来会有跳跃（steppy）和抖动（judder）感。高帧率（HFR）技术可以使观看体验更加平滑，对于数字天文学演示而言尤其如此。对于预渲染的影片而言，标准的全景球幕帧率是 30 fps，实时渲染图形的标准帧率则是 60 fps。120 fps 以上的更高帧率，以及 3D 立体全景球幕，在技术上都是可行的，这需要影院采用额外技术来播放 3D 影像，以单眼 60 fps 的帧率播放完整的球幕原版序列的左眼和右眼帧。
This resolution and HFR requirement can be a significant burden for producers especially when combined with fulldome 3D. 8K × 8K frames are four times the resolution of 4K × 4K frames, which are also four times the resolution of 2K × 2K (essentially HD) frames. This means that 8K frames take four times longer to render and are four times larger than 4K × 4K frames. HFR shows that play back at 60 frames per second require twice as many frames as a standard 30 fps show. Adding 3D doubles the number of frames that must be produced, since 3D requires a complete set of frames for the right eye and for the left eye. The math on this is staggering: each 8K × 8K file (uncompressed) would be approximately 192 MB. For 60 fps 3D, this would require 120 8K frames per second. At 192 MB per frame, this translates to 23 GB per second. The average length of a fulldome show is 22 minutes; therefore, a 22-minute 8K 3D 60 fps per eye show would require 79,200 frames per eye at a total data size of 15 TB per eye (uncompressed) or a total of 158,400 frames and 30 TB for such a show. This assumes no image compositing, or one layer per frame, which is not realistic given standard “layered” production workflows. Producing layered elements for each frame would make the production process even more daunting.
With this said, the results of 8K fulldome production can be truly astonishing, surpassing any other conventional form of media in terms of resolution, immersion, and raw impact. GPU based rendering technology has made 8K production more possible, but it is far from practical or cost effective at this time.
Traditional cinema has a well-established collection of visual storytelling tools. These include the following:
传统电影拥有一套成熟的视觉叙事工具集，包括以下内容：
Editing 剪辑
Composition 构图
Camera Movement 摄影机运动
Actor Blocking 演员走位
Color 色彩
Light 光照
Camera Angles 摄影机角度
Lenses 镜头
Special Effects 特效
Many, many others 许多其它工具
Over 100 years, filmmakers have refined the uses of these techniques within the constraints of the medium.
在过去的一百多年里，电影制作人在这一媒介的限制内，不断精进每项技术的应用。
Two key attributes define the essential capabilities of cinema: the frame itself and editing. In traditional cinema, the frame itself is one of the most powerful tools filmmakers use to direct the audience’s attention and guide them through the story. The frame inherently restricts the audience’s view and perspective and only presents what the director wants the audience to see at any given time. Combined with skillful editing, the visual story ebbs and flows according to the director’s vision, focusing the audience exactly where they want it shot by shot, and scene by scene throughout the film. In traditional film, the frame confines, restricts, and contains the imagery within each shot. The art of editing transitions these pictures rhythmically and structurally, using shots as building blocks of the visual story. Editing combines shots with varying fields of view (wide shots to close ups), and the frame itself helps define the type of shot within it, since the subject in a close up fills more of the frame than the subjects in a wide shot.
Also, in traditional cinema, the audience is looking at the film voyeuristically. Through empathy and emotional resonance, the audience may feel very connected to the experience, but they never feel that they are actually inside the film, since it clearly exists out in front of them. Surround sound and more advanced dimensional sound systems like Dolby Atmos, Fraunhofer, Iosono, Barco Auro, IMAX 12.0, and others envelop the audience with sound to help create a stronger sense of immersion.
With all of its storytelling power, however, traditional cinema cannot place the audience inside a scene.
虽然传统电影拥有强大的叙事能力，但它无法将观众置于场景之中。
全景球幕影片制作的挑战 FULLDOME FILMMAKING CHALLENGES
Fulldome filmmaking is inherently immersive, as previously described. The audience starts out inside the experience and remains immersed throughout the film. This effect can be very powerful, but there are many challenges to storytelling that this sense of presence and the medium itself creates.
Here are just some of the challenges to storytelling in domes:
以下是在球幕中叙事的一些挑战：
演员在银幕上的大小 Size of Actor(s) on Screen
Because the dome evokes such a strong sense of presence inside a place, there is a natural expectation that objects should look the way they would appear in real life, especially when employing subjective camera techniques that anchor the audience in the scene as if they are actually there. Humans depicted on screen need to look scaled properly against their environments on screen, which is also true of traditional cinema, but in subjective scenes, the actors on screen should also feel properly sized relative to the theater screen and the audience. If a human on the dome appears too large aesthetically, they appear giant, and if they appear to be too small aesthetically, they seem to miniaturize. This seems particularly problematic, since dome sizes vary dramatically from venue to venue. In practice, however, if the human character feels right on the dome, any dome, the scale of the actor will work regardless of the dome size, since the angle of light emanating from the object on the dome will be the same whether the dome is very large or relatively small. Larger domes expand out equally in all directions, so everything scales up equally, appearing the same relative size. The converse is true of smaller domes. An interesting phenomenon, but it works consistently because of the applied physics of light and the equal curvature of the dome despite its size.
The team that produced the fulldome show for New York’s Madame Tussaud’s Wax Museum in 2000, did extensive compositing tests to determine the specific placement and scale of the on-screen actor who played the hansom cab driver who guided the audience through the history of New York City. It was key to the effectiveness of the attraction that the cab driver appeared to be the proper size relative to the immersive space.
Cuts can work, but not in a traditional sense. Scenes built through inductive editing (lots of close ups) don’t usually work. The dome canvas creates too large and too wide of a view to achieve the same “restricted” view that closer shots provide in cinema when they are contained by the frame. There is no Psycho “shower scene” that exists in the fulldome medium any more than such a scene could exist on a live theater stage, at least not using traditional cinematic editing techniques.
Directors must creatively employ other techniques like camera movement, blocking and choreography in front of the camera, fades to black, and other methods borrowed from the world of theater to keep the audience oriented and anchored inside the virtual space as much as possible.
There is a lot to see on the dome canvas that spreads across the audience’s field of vision. Fast cutting from one shot to another doesn’t give the audience adequate time to see what is in a given shot, since it is replaced quickly by another shot. According to Wired Magazine (https://www.wired.com/2014/09/cinema-is-evolving/) modern Hollywood films have an average shot length of 2.5 seconds. This type of cutting is way too fast for fulldome. Also, consider the similarity to a theatrical experience that the dome provides. Audiences can’t get up and rapidly shift from one seat to another every 2.5 seconds in the theater. That would be absolutely exhausting and ridiculous. Even with objective scenes on the dome, longer shots work much more effectively on the dome than shots that are less than 3 seconds in duration. With that said, a good rule of thumb is the less complex the shot, the shorter it can be on the dome (within reason). The more complex the shot, the longer it needs to be for the audience to perceive it properly.
In traditional cinema, fast editing can add to the intensity and create a percussive rhythm to a scene; not so in fulldome. This type of editing is absolutely disorienting to the audience unless used sparingly for effect.
Think of the dome as a portal to put the audience inside a shot that physically surrounds them. There is a complete and large hemispherical canvas of information for the audience to absorb and comprehend. Shots that are too short don’t give the audience time to orient themselves in the space of the film. If the goal is disorientation, then this technique can be extremely effective, but if disorientation is not the intent, it is best to give the audience time to be absorbed into the scene.
Motion must be slowed down to feel correct and natural on the dome, compared to cinema and television, since a dome has more real estate to cover. Think of the fulldome canvas like being on the front row in a movie theater with a giant screen. Super-fast movements can be overwhelming. On a computer monitor, for example, if an object moves from one edge of a 27” screen to the other edge in 2 seconds, that movement may feel somewhat slow (a speed of 13.5” per second), but on a 60’ wide dome that same 2 second movement would move at a speed of 30’ per second: dramatically faster perceptually. This is not to say that fast movement isn’t possible on the dome. It is. It’s just vital to understand the acceleration effect the medium provides and to compensate for this accordingly.
The canvas itself is circular and distorted as if by a fisheye lens. As stated previously, this circular frame is often called the dome original or dome master. The center of the circle represents the top, or zenith, of a dome hemisphere with 0º tilt, or the spot directly above the audience. For tilted domes, this spot moves forward by the exact number of degrees the dome is tilted forward. For example, a dome tilted forward 30º will have the zenith point tilted 30º forward of the actual point that is directly above the audience’s heads. The bottom of the circle represents the front of the dome, the top of the circle represents the back of the dome, the left side of the circle is audience left, and the right side is audience right.
One of the challenges of the production workflow when creating dome originals is that they are always visually distorted. Pixels compress near the zenith of the image and they stretch dramatically near the edges of the circular frame. Only when these images are projected inside the dome do they properly undistort by mapping to the three-dimensional geometry of the hemisphere itself.
It is imperative to screen material on the dome to determine and evaluate proper scale, placement, sense of motion, shot duration, etc. since these phenomena are virtually indistinguishable when watching playback of a fisheye preview on a computer monitor.
More experienced fulldome creators can get a good sense of how a scene is playing from a fisheye preview, but even seasoned professionals can miss important details if they don’t view their work on a dome.
Interestingly, the advent of Virtual Reality has contributed to the development of innovative new camera technology that allows filmmakers to capture real world imagery in 180º and 360º. In fact, there are significant parallels in processes between VR production and fulldome production. VR often requires a completely spherical image, but fulldome only requires half of that, a 180º hemisphere. Spherical VR cameras used for fulldome production offer filmmakers the flexibility to adjust the tilt and orientation of the spherical imagery for optimum placement on a non-tilted and a severely tilted dome without having to reshoot content for display on multiple types of dome configurations. The appropriately formatted hemispherical dome originals can be extracted from spherical equirectangular frames.
There are modern cameras that have up to 4K vertical resolution, and for a majority of domes these offer enough resolution for the live-action imagery to look reasonably good on a 4K × 4K digital dome. The problem is that for the images to look as crisp and as realistic as possible, they really need to be produced at a higher resolution than 4K to allow pixel sub-sampling to optimize each pixel on the dome. Furthermore, 180º fisheye lenses cut off the horizon if they are pointed straight up during a shoot. Fisheye lenses with fields of view over 200º are needed to see below the lens and photograph the horizon below the camera.
A single 8K camera can be mounted on a Nodal Ninja, which is a camera mount that has built in nodal points around a shared center point, allowing a scene to be photographed multiple times from a fixed position. The camera points in a different direction for each take as the camera angle shifts from nodal point to nodal point until capturing all angles of a hemisphere or complete sphere. This technique can produce phenomenally high-resolution master frames: 16K × 8K spherical, but without computer-controlled tripods or dollys, all shots captured this way must be stationary with no camera movement at all. This technique provides ultimate resolution at the cost of camera movement.
Multi-camera rigs tend to integrate short focal length lenses in order for each camera to see the widest possible field of view. This reduces the total number of cameras needed for the rig and creates fewer recordings that will have to be stitched together to create the hemispherical or spherical imagery. Spherical cameras come in all shapes and sizes, from the GoPro Max with two fisheye cameras mounted back-to-back, to larger rigs containing multiple RED, ARRI, or other professional cameras in a spherical array, and everything in between including the Insta360 Pro camera, which is essentially a point-and-shoot spherical camera with multiple lenses and cameras imbedded around a sphere. Companies such as Radiant Images in Los Angeles specialize in these types of camera systems, offering a varied assortment of choices for purchase or rental.
多摄影机装置（multi-camera rigs）往往集成了多个短焦距镜头，使得每个摄影机都能看到尽可能宽广的视野。这种组合减少了装置所需的摄影机总数，也减少了为创建半球形或球形影像而需要拼接在一起的影像数量。球形镜头摄影机有各种形状和大小，从 GoPro Max 这样背靠背安装两个鱼眼镜头的设备，到包含多个 RED、ARRI 或其他专业摄影机的大型装置，以及介于两者之间的 Insta360 Pro 相机，它本质上是一种将多个镜头和摄影机嵌入一个球体周围的即拍式（point-and-shoot）球形摄影机。洛杉矶的 Radiant Images 等公司专门经营这类摄影机系统，提供了购买或租赁的多种选择。
Stitching software to combine images from each camera into a unified spherical image is often rather primitive and requires considerable physical tweaking and painstaking adjustments to optimize and master an image that looks seamless. Additionally, the physical layout of the cameras in the spherical rigs creates dead zones in the spaces between cameras in the rig especially if a subject comes too close to the camera rig. Choreography in front of a spherical camera must be carefully orchestrated to keep actors and key objects far enough from the rig to prevent them from splitting apart visually across the seams between the cameras in the array.
Lens selection is also extremely limited for fulldome live action photography. Traditional cinematographers rely on a variety of lenses to produce visual aesthetic effects to structure the depth of field (how much of the background is in focus) and field of view (how much the audience can see inside the frame). In fulldome photography, the audience generally sees everything, since the imagery inherently surrounds them, and generally everything is in focus deep into the frame.
Of course, filmmakers may also choose to shoot live action elements in front of a green screen with traditional video cameras, and then composite these assets into hemispherical CG environments using fulldome production tools produced by Evans & Sutherland, Sky-Skan, Multimeios, and others.
Filmmakers must consider and navigate the complexities of the fulldome technical workflow in order to deliver shows on time and on budget.
电影制作人必须考虑并测试全景球幕技术工作流程的复杂性，以便在预算内按时交付成片。
Figure 7. An 8mm focal length fisheye lens. Source: Photo by Jud McCranie distributed under CC-BY-SA 4.0
For single camera production, there is one type of lens: the fisheye (Figure 7) that captures a single view that is 180º to 250º wide. Fisheye lenses are the least flattering lenses for people since their extreme optics will severely distort objects that come too close to the lens, creating an unflattering effect known as foreshortening (Figure 8). The part of an object that is close to the camera appears extremely large while parts of the object further away appear unnaturally small. This is especially disastrous with human faces, since noses and chins can become enormous relative to the rest of the face when a person is too close to the lens.
Figure 8. Foreshortening effect caused by being too close to a fisheye lens Source: Dr. Denny Vrande č i ć, distributed under CC-BY-SA 4.0
Fisheye lenses also have infinite depth of field, so everything is always in focus, removing one of the classic cinematic techniques that shallow depth of field allows: blurry backgrounds. Soft focus in the background and sharp focus on the subject draws the audience’s attention to the subject that is in sharp focus and away from the soft-focus background. This technique facilitates moments in which the director wants to amplify the emphasis on the subject and minimize distractions. Live action lenses for digital fulldome can’t do this, so directors must employ other techniques to direct the audience’s focus and attention.
Current digital transforms developed to stretch and warp 4: 3 aspect ratio produced by 15/70 IMAX film enable flat screen imagery to cover 80% of the dome (as IMAX Dome systems have done since the 1970s). Live action footage can be shot with any lens in 4: 3 aspect ratio and then warped to the dome. This technique leaves a 20% gap of black in the back of the dome that needs to be filled in with CGI or just feathered to black. This workaround allows cinematographers access to essentially any lens, letting them break free from constantly having to choose a fisheye or wide-angle lens.
A dome is a very challenging environment for projection. Because light falls on every part of the dome, the light from the back of the dome bounces back toward the front, scattering light into the dark areas of the scene, thereby reducing the contrast. This phenomenon called cross-reflection has a similar effect to what happens in a movie theater when the house lights come on during the credits roll at the end of the film. Blacks on the screen turn gray, and the contrast ratio of the image drops dramatically. LED domes are self-illuminating, solving the cross-reflection problem, producing bright images with phenomenal contrast. Currently LED domes are quite expensive, but as with all new technology, its price will drop and become more affordable over time.
球幕是一种极具挑战性的投影环境。因为光线会落在球幕的所有位置，球幕后部的光线会反射回前方，将光线散射到场景的暗区，降低画面对比度。这种现象被称为交叉反射（cross-reflection），与电影放映结束时观众席灯光亮起、银幕上黑色变灰、画面对比度急剧下降的效果类似。自发光的 LED 球幕解决了交叉反射问题，可以产出具有惊人对比度的明亮图像。目前 LED 球幕相当昂贵，但就像所有新技术一样，其价格随着时间的推移会不断下降，使场馆更加负担得起。
There is also only one perfect seat in the dome, and that is right in the middle, right under the zenith. This is the only place where all the images perfectly unwrap from their distorted dome original to look straight and undistorted on the dome. Anywhere else in the dome, vertical lines will curve. It’s simply a physics problem. Domes are compound curves, and they are curved everywhere. Domes warp everything, so directors need to be careful of straight lines. They won’t stay straight. This isn’t always a problem, but it is something to consider when designing an experience for an audience.
数字全景球幕中的关键视觉叙事工具 KEY VISUAL STORYTELLING TOOLS FOR DIGITAL FULLDOME
The following list of tools work extremely well to help craft visual stories in the fulldome medium. By using these techniques separately and together, directors can tell amazing stories despite the aforementioned limitations of the fulldome medium.
The arrangement of items across the fulldome canvas is one of most powerful resources in the director’s toolkit. There is an expansive world to paint both in terms of the imagery that surrounds the audience, but also what exists overhead. The fulldome canvas is surprisingly supple and can easily take the shape of the environment placed around it. Low ceilings and close walls can simulate a confined space; conversely tall trees stretching high above the audience can create the strong sensation of an expansive space. Many of the other tools below, except for the last two, work within the composition of the scene to create show’s visual aesthetic. More than virtually any other medium, fulldome does not have an obvious point of focus or place to look; therefore, it is imperative to skillfully and constantly guide the audience’s attention and visual focus throughout a show. The techniques below are some of the time-tested methods that can do just that.
Moving the perspective of the entire scene, crafting dynamic and changing compositions by moving the camera, provides visual interest to scenes, allowing them to develop over time and stay on screen longer without the need for cuts. The more the scene can change within the shot and the camera’s choreography, the more the audience will stay anchored within the location or place of the scene.
Camera movement must be handled skillfully, since the audience often feels that they are actually moving with the camera through the scene. It is quite a powerful kinetic connection that immersion creates in the audience.
Just like any technique this can be underused and overused. Some notable fulldome films are comprised of one continuous shot over their 20 to 25-minute running time. This can be quite effective, but also has the unexpected complication of forcing the camera path over time into dead moments that have to be enhanced to liven them up (much like long stretches of open highway on an endless road trip) or to restricting your view to what can logically fit along that unbroken camera path. Intentionally leading with a technique or gimmick can diminish the overall creative possibilities and highlight the technique at the expense of the impact of the overall experience. Everything in moderation, unless there is a compelling creative reason to lean heavily into one technique or another. It’s a delicate artistic balance that the director must constantly evaluate.
Just as in a stage play, blocking helps define the scene’s action and the actors’ characters. Whereas a pure theatrical experience is limited by the confines of the available performance areas inside the theater, in a fulldome show, actors can appear and move anywhere within the immersive canvas. This needs careful balance as well, but in a dynamic shot, even if the camera is stationary, actors moving toward and away from the camera or moving through the scene can help guide the audience’s attention within a frame that has no clear director or point of focus. As humans, we like to look at other humans, and our eyes follow them throughout a scene. This is a powerful way to guide the audience’s gaze inside the frame, which is more crucial than ever considering the inability to use traditional cinematic editing to restrict the audience’s view.
The human eye also follows movement. Objects that move against others that are stationary grab attention. Something that moves causes humans to want to know what is moving, where it is going, and what it may be up to. This could relate to simple survival skills or basic curiosity. Either way, object movement inside the frame is a highly effective way to guide the audience’s attention through the expansive fulldome canvas.
Humans seem to have a desire for order and organization. Even a picture that is slightly off of perfect level screams out to be adjusted. Visual balance in a fulldome frame is no exception. Similar objects or those that have a symmetrical layout on screen are easily ignored, but anything that stands out because of its size (smaller or larger than other objects) or because of its visual isolation within the frame draws focus. This orchestration of visual balance is not nearly as powerful as movement or a human in the frame, but the choreography of visual elements and the balance of their visual symmetry and asymmetry can cause viewers to look at the objects that stand out. This is pure visual composition and the creation of an area of interest inside the fulldome frame and is most analogous to composition principles in conventional cinema.
In a similar way, color asymmetry can cause the audience to focus on colors that look different or draw attention. For example, a green object will pull focus in a scene that is largely blue. Humans are very aware of differences, and unique colors in a scene will stand out. Also, warm colors (reds, yellows, oranges) command attention and push to the foreground while cool colors (blues and purples) recede and move to the background. Greens are neutral and can serve either purpose. White and black can switch roles based on the surrounding colors. In other words, either black or white can be dominant or recessive based on the context of the scene. With that said, white tends to grab our attention more than black if all other factors are neutral.
Human eyes are basically light receptors. Light activates sight. Light on objects in a scene makes people look at them. It’s the same reason people follow the beam of light from a flashlight in the dark, even if it isn’t illuminating anything interesting. Light directs the viewer’s attention very powerfully. Once again this is a time-honored tool from the beginning of cinematography and theater. Lighting is a very useful too that can guide the audience attention throughout a show.
As much as humans like to think they are very complex and sophisticated, they default to what is simple and easy to understand. The human eye may be fascinated by complex imagery but will always try to simplify it or find the least complex place to look. This is why tunnels work so well on the dome. The audience looks right down the center of the tunnel to see where it is leading. The choreography and interplay of simple and complex visual structure can effectively show the audience exactly where to look in a scene. This can be everything from a tunnel to an alleyway blocked by tall buildings on both sides, to the tracks on a rollercoaster.
Vectors are imaginary lines that point in certain directions within a frame. These cause the audience to look in the direction the vector is pointing or moving. Vectors on the dome can be caused by graphical composition where a line or curve forms from the spatial arrangement or simple patterns among a complex scene as described above. These are known as graphic vectors, which tend to be subtle. Other vectors that point in a certain direction, like an arrow, a street sign, or a finger pointing at something are called index vectors. These are a bit more powerful than graphic vectors. Finally, there are motion vectors where an object is moving in a direction, and the audience watches with interest where it is going. Motion vectors have the most weight and ability to direct the audience’s eyes through the scene.
This is a specific type of vector generated by an actor looking in a particular direction on the screen. Wherever the actor looks, the audience is compelled to look as well. Part of this likely springs from innate curiosity; part may result from the fact that looking at something is a less obvious form of pointing. For this vector to work most effectively, the actor should look at something in the scene. This way the audience can follow the actor’s gaze to a landing point within the frame.
There is little offscreen space in fulldome, since the frame is so vast, but it is possible to use clever editing to connect the actor’s eye line from inside one shot into an object in the next shot.
As previously discussed, editing is possible and often necessary in digital fulldome. Editing allows directors to shift between subjective and objective scenes and to change the audience vantage point within an objective scene. Editing can also transition between subjective sequences to string larger sections of a film together.
There is no shortage of authoritative reference material for cinematic editing techniques, and there is neither space nor need to delve into these in this chapter. Suffice it to say that most time-honored principles of traditional editing still apply to digital fulldome, even if the frame or the immersive medium itself does not respond the same way as a cinematic frame that sits out in front of the audience. Experimentation and careful shot/sequence planning is essential to discover what can work the most effectively in a given project.
When used most effectively, cuts can become largely invisible and feel completely organic to the visual story. This places pacing and scene sequencing back in the hands of the director even in the fulldome medium.
Audio cues are extremely effective in grabbing the audience’s attention, especially in moments where the director wants the audience to look up, to the sides of the screen or behind them toward the back of the dome. Even though fulldome is an immersive medium with 360º imagery, when an audience is seated, they typically look forward unless there are reasons for them to look elsewhere. Sound cues are an organic and elegant way to help direct the audience’s attention to different parts of the dome. This technique must be used gently, because it is uncomfortable for the audience to require them to move their bodies around to physically look behind them, especially when they are in a seated position. Better to use sound in the back of the dome to bring in something that quickly moves to the sides of the dome or across the zenith, such as a spaceship entering from behind and moving overhead. In this example the audience will only have to look up as the ship is passing above them.
有效的全景球幕叙事示例及分析 EXAMPLES AND ADVANTAGES OF EFFECTIVE FULLDOME STORYTELLING
The immersive fulldome canvas is not inherently better than traditional flat screen cinema, but as explored in this chapter, it is inherently different. In addition to the variety of specific aesthetic effects immersive content provides, fulldome creates a unique connection with the audience. They feel like they are inside the story, sometimes in first person point of view (POV) and other times in third person POV, but in both cases they are watching from inside the story as opposed to watching as a voyeur in traditional cinema where the audience is outside the experience looking in through the frame.
There are a host of films produced specifically for the fulldome medium that have leveraged these principles effectively within certain scenes and in some cases throughout the films.
有一系列为全景球幕特别制作的影片，将上述原则有效地应用在部分场景甚至全片当中。
Microcosm: The Adventure Within: (Daut, 2002). The first science fiction and non-astronomical fulldome film ever created (written, produced, and directed by the author) was also the first time that a fulldome film moved from entirely a first person POV and made extensive use of cinematic editing to propel the story and change audience perspective in the immersive space. What started as a grand experiment actually helped define and clarify modern fulldome cinematic techniques.
The conceit of the film placed the audience inside a human-crewed submersible that could shrink down to microscopic size for injection into the bloodstream of a dying patient. After an introduction that set up the scenario, the audience finds themselves inside the submarine’s cockpit looking out the front window with controls and hardware of the submarine’s interior surrounding them. The show established this as the first person POV for the entire show. Rather than anchoring the entire experience inside the sub, the perspective cuts to a view outside the sub, watching the scene from within the scene from a third person POV. This change in perspective within a scene enabled editing to allow the viewer to move easily from one POV to the other; in fact, from the third person POV, cuts from shot to shot were easier to achieve because of the aesthetic distance created by the third person POV. This continuous sense of presence in the scene creates a dynamic effect that could not be possible in a traditional cinematic experience.
Microcosm: Stars of the Pharaohs: (Murtagh & Daut, 2004). This film broke ground with its stunning recreations of temples and tombs of ancient Egypt including the temple of Denderah (built in 3D as a photorealistic representation of how it exists today in its partially ruined state) and the temple of Luxor (built in 3D as a depiction of how it may have looked 3,000 years ago). By establishing a first person POV, the audience was allowed to travel inside these amazing locations in such a way that they felt transported not only to Egypt, but also back in time on a walk-through of these astonishing destinations. Traditional cinema does not allow this sensation of transportation inside virtual worlds.
《微观宇宙：法老之星》 (Murtagh & Daut, 2004)。这部作品通过精致再现古埃及的寺庙和墓穴，如丹德拉寺庙（用 3D 打造，呈现其今日残破的真实面貌）和卢克索寺庙（用 3D 展现其 3000 年前的可能样貌），开辟了球幕影像的新境界。通过采用第一人称视角，观众仿佛被带到了埃及，不仅跨越了空间，更穿越了时间，亲历了这些令人叹为观止的地点。这种仿佛置身虚拟世界的感受，是传统电影无法提供的。
Dawn of the Space Age: (Sip, 2007). Telling the story of mankind’s efforts to take the first steps into space, the competition between the USA and the Soviet Union, the moon landing, and humanity’s future in space, this show uses the third person POV throughout the entire film. This is a perfectly appropriate technique for this particular narrative, since the filmmaker invites the audience to become a bystander inside defining moments of the space race, but not to take the first-person perspective of those pioneers who changed mankind’s destiny in the universe. The audience was never allowed to become the historic space pioneers, but only to be with these pioneers to experience these seminal moments with them as never before. This third person perspective allowed freedom of editing from one shot and one scene to another, since the audience never perceived that they were the people in the story, but only very close observers.
Dream to Fly: (Heavens of Copernicus & Majda, 2013). Just before the midpoint of this film, the filmmakers place the audience inside a conventional theater facing a curtained proscenium that towers in front of them. This motif riffs on the theatrical nature of fulldome immersion, previously discussed in this chapter, and then pulls set pieces off the stage and moves them around the audience space, everything from sketches on parchment paper and aerodynamic equations to prototype planes that fly around the interior of the theater. This sense of theatrical immersion does not directly translate into flat screen cinema experience, and is therefore unique and completely fitting for use in the fulldome medium.
《追梦飞行》 (Heavens of Copernicus & Majda, 2013)。在影片的中段之前，制作者巧妙地将观众置于一个传统剧院之中，面朝一个挂着帷幕的巨大舞台。这种设计巧妙地回应了全景球幕沉浸体验的戏剧性质，接着将舞台上的道具——从羊皮纸上的素描，到空气动力学公式，再到围绕剧场内飞行的飞机原型——移至观众周围，使之成为场景的一部分。这种戏剧化的沉浸体验是传统的平面电影所无法直接复制的，在全景球幕媒介中展现了其独特且完美适配的特性。
Expedition Reef: (California Academy of Sciences & Wyatt, 2018). The film begins with a scene inside an aquarium facing a large floor-to-ceiling glass tank containing a coral reef environment. The audience is in the building looking at the aquarium, an interesting play on “looking at” versus “being in” a scene. As the camera moves toward the glass on tank, the interior of the aquarium melts away and blends into the coral reef, placing the audience in the middle of the open ocean on a journey of exploration into the biodiversity of the coral reef. This is a powerful immersive storytelling technique that would not be possible in traditional cinema.
《珊瑚礁探险》 (California Academy of Sciences & Wyatt, 2018)。影片开场是在一个水族馆内，观众面对着一面巨大的、从地面延伸到天花板的玻璃水族箱，里面仿佛包含了一个珊瑚礁的小宇宙。这里展示了一种独特的视角游戏：观众是在建筑内部观察水族馆，形成了一种“观看”与“置身场景中”的对比。随着镜头逐渐推近水箱玻璃，水族馆的界限开始模糊，最终与珊瑚礁景观融为一体，观众仿佛被带入了无垠海洋的深处，开启了对珊瑚礁生态多样性的探索之旅。这种沉浸式的叙述方式，在传统电影里是难以实现的。
The show continues with long-duration shots featuring complex camera moves sweeping through the reef, pausing to introduce new creatures and story points to the audience, anchoring them in a first-person POV with a highly choreographed camera paths that efficiently move from one story point to the next with very infrequent cuts and scene transitions. The opening shot lasts nearly five minutes, starting underwater, breeching the surface, hovering over the shoreline, then flying out into space to look at the Earth holistically. This type of sweeping, complex shot would not have the same sense of presence in traditional cinema and would likely need more edits to preserve the pacing for the flat screen experience.
The fulldome canvas is huge. If a picture is worth a thousand words, then a fulldome picture is worth ten thousand words. Directors should strongly consider designing the visual story first, almost as if crafting a silent film. They should ask what story the visuals are telling and how many words are actually needed to complement the scenes. It is not necessary to use words to state what is obvious in the visual story.
Action, camera movement, blocking, light, color, composition, and the other visual storytelling techniques described should build each shot and sequence to guide the audience through each beat of the story.
Choosing whether to design an objective or subjective scene or sequence is most directly analogous to the throttle or gas pedal for immersive storytelling. Subjective scenes, where the audience feels like they scene is happening to them, are the most potent and impactful. Objective scenes where the audience feels like they are close to an event as just an observer or bystander have slightly less impact and offer directors more opportunities to cut.
Subjective scenes, for example place the audience inside a car in the driver’s seat (maximum emotional and kinetic impact), whereas objective scenes place the audience outside the car to witness what another driver is doing (one step removed from personal participation, but still strong involvement). Subjective scenes anchor themselves in real time, since they mimic real experiences. Temporal flow must remain natural and uninterrupted. Objective scenes can jump between places and time can be left out when an edit takes place.
Structuring a balance between objective and subjective scenes effectively can allow directors to most powerfully guide the audience through a visual story while balancing the thematic arc and emotional impact.
通过有效平衡客观与主观场景，导演可以在不牺牲情感冲击力的同时，更加有力地引领观众穿越视觉叙事。
精心设计时长较长、复杂度较高的镜头 Choreograph Longer Shots with Higher Degrees of Complexity
Think of the medium as immersive theater with an audience that can move around the theater to a limited degree. In digital fulldome, when the perspective moves, the audience moves. Rather than rely on editing to structure and build a sequence, the dome more natively wants to present shots that are highly choreographed from camera movement, to blocking, to art direction, to the arrangement of items in the frame, to the dynamic, unfolding interest generated by a longer shot that is constantly changing and developing as the audience moves through the scene.
The more complex the scene, the more it holds the audience’s interest, and the longer it can be sustained without a cut. This is an extraordinarily important technique to learn and implement.
以媒介特性和观众体验为核心进行剪辑 Edit with the Medium and Audience in Mind
Don’t rely on editing to build scenes the way it can in traditional cinema.
不要像在传统电影中那样，依靠剪辑来构建场景。
Don’t rely on close ups, master shot, medium shot, over the shoulder shots, since the dome is largely frameless, and these shot descriptions can’t apply in the same way. There is no direct correlation to this type of shooting or sequence editing. The scenes are either objective with limited editing or subjective with little or no editing except for transitions from one scene to another. Craft shots that are visually interesting and transform into different looks as the camera moves and as subjects move within the fulldome frame.
发挥球幕变化自身的能力 Leverage the Dome’s Ability to Transform Itself
The canvas itself is very big. Make it smaller when needed or let it be giant. The shape of the dome mimics the sky or infinite space. Its geometry does not make itself obvious to the audience; instead, it seems to transform its shape and size to perfectly match and reproduce any environment projected onto its surface. That makes the dome somewhat magical, when it can use these “superpowers” to mold itself into a tiny passageway through an underground cavern or a giant cathedral with 60’ vaulted ceilings. Both are entirely possible as is everything in between. The aesthetic feeling of each space can also help shape and convey the story.
Don’t tell fulldome stories: tell stories. Don’t let the medium itself force story choices that don’t best serve the story’s intent. This means that the story may require live actors inside the dome theater that interact with moments on the dome. This may mean that set pieces, props, a preshow, or seamless branching between story segments may be required. Let the story shape the experience NOT the medium. Definitely consider all the medium’s uniqueness qualities, but don’t be confined by the restrictions of the medium itself.
A number of feature films have begun experimenting with longer, highly choreographed shots that illustrate sensibilities that work perfectly for fulldome production.
The film opens with a single shot that runs for 13 minutes and 7 seconds with no cuts. This shot is exquisitely choreographed with camera movements, actor blocking, staging of the spacecraft in the scene. This groundbreaking shot is extremely dynamic and provides masterful pacing and remarkable storytelling (Cuáron & Heyman, 2013).
The first boxing match in the film is shot with a moving camera and no cuts at all. Once again, the shot never loses energy and does not suffer from a lack of fast edits. The scene constantly changes (Winkler, et al., 2015).
The director employs a long uninterrupted shot in the middle of the film where a battle breaks out at a casino where unsavory activities are taking place. Another excellent example of camera and actor choreography in a sustained, dynamic shot (Fiege & Coogler, 2018).
Much of this film’s entire running time is one single shot with no observable edits (but of course the film was not shot in one take). Clever edits are hidden as the camera moves across a neutral background. More inspiration for choreography on the dome (Iñárritu et al., 2014).
Don’t rush your shots. Give the audience time to experience the scene. Base shot duration on complexity of the scene
Don’t forget to move the camera
Don’t forget the audience moves with you. Guide them through the experience like an expert docent at a museum or historical site
不应让技术主导剧情；
不要移动太快；
镜头不要太急，让观众有时间沉浸在场景中，镜头的长度应由场景的复杂度决定；
记得移动摄影机；
记住，观众会随着你的镜头移动。假装你是博物馆或历史场所的资深讲解员，引领他们深入体验。
结论 CONCLUSION
Cinema and dome cinema grew up in very different ways. Their technology and their origins have led to their current maturity and the depth of content that is possible in each medium. Similar tools exist in both mediums, but their use is very different. Education, not entertainment, was the origin of digital fulldome; whereas, cinema could educate, entertain, and explore any genre from its inception. Storytelling and entertainment experiences are not only possible, but game-changing in digital fulldome, and effective storytelling techniques and best practices exist and are continuing to expand.
The fulldome medium has the potential to attract large audiences and to deliver a mind-blowing VR shared theater experience for audiences of all ages and even eclipse the storytelling capabilities of Hollywood films, potentially kicking off a revolution in fulldome filmmaking and even the establishment of a dome theater network across the world. The future of fulldome is yet to be written, but its potential is absolutely unlimited.
California Academy of Sciences. (Producer), & Wyatt, R. (Director). (2018). Expedition Reef [Fulldome Film]. United States: California Academy of Sciences.
Cuar ó n, A., & Heyman, D. (Producers), & Cuar ó n, A. (Director). (2013). Gravity [Motion Picture]. United Kingdom, United States: Warner Bros. Pictures.
Feige, K. (Producer), & Coogler, R. (Director). (2018). Black Panther [Motion Picture]. United States. Marvel Studios.
Heavens of Copernicus Productions (Producer), Majda, P. (Director). (2013). Dream to Fly [Fulldome Film]. Poland: Heavens of Copernicus Productions.
I ñá rritu, A. G., Lesher, J., Milchan, A., & Skotchdopole, J. W. (Producers), I ñá rritu, A. G., (Director). (2014). Birdman [Motion Picture]. United States. Fox Searchlight Pictures.
Lambert, N., & Phillips, M. (2012). Introduction: Fulldome. Digital Creativity, 23(1), 1–4. doi: 10.108 0/14626268.2012.666980
Marche, J. (2005). Theaters of Time and Space: American Planetaria, 1930-1970. New Brunswick, NJ: Rutgers University Press.
Michael, D. (Producer), & Michael, D. (Director). (2002). Microcosm: The Adventure Within [Fulldome Film]. United States: Evans & Sutherland.
Murtagh, T., & Daut, M. (Producers & Directors). (2004). Stars of the Pharaohs [Fulldome Film]. United States: Evans & Sutherland.
Sip, R. (Producer & Director). (2007). Dawn of the Space Age [Fulldome Film]. Netherlands: Mirage3D.
Winkler, I., Chartoff, B., Winkler, C., Winkler, D., King-Templeton, K., & Stallone, S. (Producers), & Coogler, R. (Director). (2015). Creed. [Motion Picture]. United States: Warner Bros. Pictures.
出处：《Handbook of Research on the Global Impacts and Roles of Immersive Media》 标题：Immersion Domes: Next-Generation Arts and Entertainment Venues 作者：Edward Lantz，Vortex Immersion Media，USA 翻译：Horace Lu
（注：键盘快捷键“w”或左侧菜单右上角按钮，可切换文章列表视图与大纲视图）
[TOC]
摘要 ABSTRACT
Large-scale immersion domes are specialized embodiments of spatial augmented reality allowing large groups to be immersed in real-time animated or cinematic virtual worlds with strong sense-of-presence. Also called fulldome theaters, these spaces currently serve as giant screen cinemas, planetariums, themed entertainment attractions, and immersive classrooms. This chapter presents case studies for emerging applications of digital domes, reviews dome theater design basics, and suggests that these venues are on track to become mainstream arts and entertainment centers delivering global impact at scale. Standard venue designs will be necessary to realize the full potential of an immersive media arts and entertainment distribution network. This chapter provides rationale for standardization of immersion domes for multi-use events spaces, immersive cinemas, and live performing arts theaters.
Spatial augmented reality (SAR) maps pixels onto physical objects to produce visually immersive or digitally augmented experiences. SAR embodiments include architectural projection mapping and digital domes (fulldome theaters) which are general-purpose immersive visualization environments capable of accommodating large groups without the need for VR headsets, glasses or goggles. Digital domes can incorporate a variety of extended reality (XR) modalities including the incorporation of live performers with interactive tracking, stage and prop projection mapping, and real-time display of cyber worlds and metaverse environments.
Because of their large capacity, high throughput and ease of use (no wearables or controllers), digital domes have the potential to become a mass medium for XR similar to cinemas. Domes and other SAR environments cannot replace VR—they are a medium in their own right. They are advanced storytelling environments capable of touring audiences through virtual worlds, scientific datasets cinematic environments and more. Walk-through and stand-up SAR environments can also be highly interactive. Digital dome venues do have a number of limitations when compared to VR. For instance, dome theaters accommodating large audiences are typically limited to a hemispheric field of view—half the visual field of VR—because large standing or seated audiences block a large portion of the audience’s visual field-of-view. And with large audiences the sense of personal agency is muted, with the experience being more akin to a tour bus ride or a large window into virtual worlds.
得益于其能够容纳大量观众、高吞吐量和易用性（无需穿戴设备或控制器），数字球幕展现出成为类似电影院那样的 XR 大众媒介的潜力。球幕和其他 SAR 环境并非要取代 VR，而是作为一种独立的媒介存在。它们提供了先进的叙事环境，能够引导观众穿越虚拟世界、科学数据集、电影环境等。步行穿越和站立式的 SAR 环境同样具有高度的互动性。然而，与 VR 相比，数字球幕在容纳大量观众时存在一定的局限性。例如，大型观众群体的球幕影院通常只能提供半球形视野——这是 VR 视野范围的一半——因为观众站立或坐着时，大量人群会阻挡其他观众的大部分视野。此外，在众多观众面前，个人的主导感会减弱，体验更像是乘坐旅游巴士或通过大窗户观察虚拟世界。
The tour bus analogy accurately describes contemporary immersion dome experiences in planetariums, giant screen cinemas and IMAX ® Domes. It would be tempting to classify dome theaters as passive group immersive cinema displays, or group immersive portals into cyberspace. However, our recent work involves the merging of live performers into real-time rendered environments, allowing performers to command and interact with the immersive world. This moves digital domes more firmly into the realm of mixed reality.
The integration of stadium seating for optimal immersion, a projection-mapped stage and foreground props utilizing SAR techniques, in combination with 360 scenes projected on the dome create a unique format that we call CineTheater ™. In addition, multipurpose dome venues without seating can serve as community hubs for connecting, creative placemaking, black box theaters, and special event venues. It is suggested that both designs will find utility and should be pursued in the development of next-generation, location-based arts and entertainment venues.
我们将针对沉浸式体验优化的体育场座椅，与利用 SAR 技术进行投影映射的舞台和前景道具，以及投影在球幕上的 360 度场景整合起来，创造了称为 CineTheater ™的独特格式。此外，这种多功能的球幕场馆在不设座椅的情况下，可以灵活转变为社区的聚集地，用于促进交流、创意场所营造*、黑匣子剧院和特殊活动场所。建议在开发下一代基于位置的艺术和娱乐场所时，应当充分考虑这两种设计，它们都将发挥其效用。
Immersion domes typically map 360 visuals onto spherical, hemispheric, or ovoid screens to deliver visual experiences similar to VR headsets but without the need for glasses or goggles (Lantz, 1997). It’s like putting a giant VR headset on a large group of people. The resulting wide field-of-view imagery can trigger brain states not usually accessible via film or other digital media such as sense-of-motion (vection), presence, scale, awe, and have been shown to communicate concepts better, create a greater interest in learning, and are more effective than a movie screen or television at conveying certain scientific concepts (Lantz, 2011).
Digital domes can provide a greater instantaneous visual field-of-view than consumer VR headsets, nearly encompassing the viewer’s entire peripheral vision. Peripheral vision is associated with vection, spatial learning, and navigation skills (Yamamoto & Philbeck, 2013). Because the audience is free to use head motion to observe immersive scenes without the need for head tracking, higher frame rates are not strictly required. Anecdotal experience with audiences indicates that VR experiences presented in digital domes are preferred by many who otherwise feel confined, experience vertigo, or have other difficulties with VR headsets.
The first digital domes were pioneered for military vehicle simulation and training, but it is the planetariums that first embraced video projection in domes for public exhibitions. At last count over 1,665 digital domes—mostly planetariums, science centers, giant screen theaters and portable domes—are documented worldwide, with nearly half in the U.S. (Petersen, 2019a). And over 335 titles have been produced for this market ranging from astronomy and space science, earth science, and more (Petersen 2019b).
While narrative storytelling has long been a mainstay of planetarium programming, with the advent of laser graphics in the 1970s and now 360 “fulldome” video graphics and VR content creation, arts and entertainment programming has increasingly found a home in planetarium programming (Lantz, 2009; Lantz, 2018).
Digital dome theaters include portable (inflatable) planetariums, smaller classroom planetariums in universities and school districts, public or private institutional digital domes largely in museums and science centers, giant screen film theaters which have converted to digital including, most recently, IMAX ® Domes. In addition, a number of theme park rides employ domes and other wide FOV screens (Fraser, 2018).
While there are now 1,665 digital domes in the world, there are only a small number of pioneering arts and entertainment dome theaters that have emerged in the past decade. It is a now a good time to establish industry standards for the coming generation of arts and entertainment domes.
There are existing industry standards that can apply to arts and entertainment domes (Lantz, Wyatt, Bruno, & Neafus 2004). The Dome Master Specification, first drafted in 2004 at the Fulldome Standards Summit in Valencia, Spain, has allowed the free exchange of cinematic digital dome programming between digital planetariums, and continues to be updated by the trade association IMERSA: Immersive Media, Entertainment, Research, Science, Arts (2014) which was founded in 2008 to foster the fulldome video format. This format was subsequently incorporated into the Digital Immersive Giant Screen Specifications, developed by the Giant Screen Cinema Association for the conversion of 70mm film theaters to giant screen digital cinema (DIGSS, 2018).
If immersion domes are to become a recognized mainstream entertainment format at scale—allowing mass distribution of immersive experiences—basic design principles must be adopted, and specific design features must be chosen and adhered to by the industry. This section focuses on some of the more critical design parameters and principles that must come into play when designing next generation immersive venues.
In general, immersive theaters typically utilize one or more video projectors to cover the entire surface of a dome screen. LED domes are also emerging but are currently limited to specialized applications such as theme parks by their high cost. Therefore, the focus of this chapter is on projection domes, with an eye towards future emissive screens such as LED. Audio systems are more flexible and do not dictate theater design to the same degree as display systems so they will not be covered in as much detail.
Most permanent dome theaters utilize one or more video projectors to cover the entire surface of a perforated aluminum dome screen. Perforated aluminum screens are nearly seamless when panels are precisely butt-seamed and are also nearly transparent to sound. This allows sound absorbing material to be placed behind the dome to mitigate unwanted reflections. It also allows speakers to be mounted behind the screen as well as fire sprinklers.
Building digital planetariums and giant screen theaters is an expensive affair. Less expensive prefabricated mobile dome theaters are a popular alternative and can be found at festivals, sporting events, conventions, and more. The most common mobile domes utilize a geodesic frame with vinyl cover and negative pressure internal fabric projection screen supported via a suction fan (Figure 1). With a UV resistant heavy vinyl cover, HVAC and an interior deck or slab, these structures can also be used as semi-permanent theaters up to 37 meters in diameter with a lifetime of 20 years or more. Air-supported domes are also popular for larger venues of 60 meters or more. These structures require positive air pressure requiring revolving doors or airlocks to maintain interior air pressure.
Dome audience viewing and interaction modalities include seated, standing and walkthrough.
球幕影院的观众观赏和互动模式涵盖了坐着观看、站立体验以及步行穿梭等多种方式。
Dome imagery can be unidirectional or omnidirectional. Each configuration has its own unique properties.
球幕影像展示可以是定向的，也可以是全方位的。这两种配置各自拥有其独到的特点。
观众安排 Audience Arrangements
There are two primary viewing configurations in immersion domes—unidirectional, where all audience members face the same direction—and omnidirectional or concentric, where audience members all face the center of the theater like sitting or standing around a campfire (Figure 2). Standing configurations often use handrails to guide visitors since the strong sense of vection can cause a standing audience to lose balance if exclusively focused on a dome presentation.
For concentric seating (a) the dome screen equator or “springline” is almost always level to the ground—typically raised just above the audience’s heads. Unidirectional seating (b) allows the dome to be tilted in the front to provide foreground. Stadium seating takes full advantage of the dome tilt to further improve viewer sight lines. More on this shortly.
Unidirectional viewing is especially advantageous for storytelling because:
The entire audience is looking in the same direction towards a “sweet spot” on the dome
Single point of focus facilitates character-based storytelling, informational text
Audience can be flown through or over terrain with everyone flying in same direction
Audience attention can easily be captured and directed to the left or right
Provides the most consistent and controlled audience experience
Stage lighting is simplified – elevated lights can be placed behind the audience
单向观看在叙述故事时具有显著优势，主要体现在以下几个方面：
所有观众的目光都集中于球幕上的“最佳视点”
统一的焦点使得角色驱动的叙事和信息性文本的传递变得更加高效
观众能够共同体验飞行或穿越地形的感觉，每个人都朝着相同的方向前进
观众的注意力能够轻松地被集中并引导至特定的左侧或右侧
这种设置提供了最为一致和受控的观众体验
舞台照明的安排也因这种布局而得到简化，因为提升的灯光可以方便地放置在观众的后方
Concentric viewing presents a number of challenges especially when delivering a cinematic experience, but there are some interesting possibilities as well:
In concentric seating the audience could be looking in any direction
The zenith (top) of the dome screen is the only common area of focus
Flying audience up towards sky or falling are powerful omnidirectional experiences
Image orientation on the zenith depends on seat location
Nearly half of the audience will see zenith image upside down
Especially problematic with text, faces and other objects dependent on orientation
Text and characters on dome generally need to be repeated around dome for all to see
Limits use of full dome images (which are the most powerful)
When flying audiences over landscapes, half of the audience will be flying backwards
Stadium seating is not possible without raising the entire dome image which reduces the dome image field-of-view for the audience.
Easily accommodates random standing audience such as cocktail party or networking
Omnidirectional seating generally results in greater audience capacity
同心式观看在提供电影体验时虽然存在一些挑战，但也带来了一些独特的可能性：
同心式座位中，观众的视线可以自由朝向任意方向
球幕屏幕的天顶部分成为观众唯一的共享焦点区域
引导观众向上飞翔或向下坠落的体验，在全向式观影中具有强烈的沉浸感
天顶上的图像方向会根据观众的座位位置而变化
大约一半的观众可能会看到倒置的天顶图像
对于文本、面部以及其他方向敏感的物体，尤其具有挑战性
为了确保每个观众都能看到，球幕上的文本和角色通常需要在屏幕周围重复显示
限制了 Fulldome 图像最为有力的使用方式
当带领观众飞越景观时，一半的观众将会体验到倒飞的效果
如果不提升整个球幕图像，就无法实现体育场式座位，这将减少观众的球幕图像视野
这种布局易于容纳随意站立的观众，适合举办鸡尾酒会或社交活动等场合
全向式座位通常能够容纳更多的观众
剧院座位 Theater Seating
Theater seats are a key component in sit-down environments. As will be seen, it is important to assure that the seat tilt provides audiences with an optimal view of the dome screen. In general, seat backs should tilt back further the closer they are to the front of the screen. Removable seats are desirable in multipurpose domes. A number of multisensory “4D” special effects can also be built into theater seats including vibroacoustic “bass shakers,” personal sound environments, seat cushion pokers and air bladders that inflate and deflate in sequence, mist, buzzers, neck and leg ticklers and motion seats.
In dome theater design we assess the degree of audience immersion by measuring the image field-of-view (FOV). The FOV is defined by the horizontal and vertical angles over which the dome image extends in a typical viewers eye when seated or standing naturally. FOV is a function of where the person is seated or standing in the dome. We use standard ergonomic field-of-view angles for both (a) seated (leaning back) and (b) standing configurations (Figure 3).
Figure 3. Standard ergonomic field-of-vision for a) seated and reclined and b) standing
We can assess the degree of visual immersion by projecting these “sight lines” from a typical viewer’s eye and measuring the vertical and horizontal angle covered by the dome image. Since the horizontal FOV is almost always at least 180 degrees, the vertical FOV is typically the limiting factor in creating a powerful sense of immersion. Ideally, the entire surface area of every viewer’s retinas would be completely mapped with pixels.
The (4a) standing configuration (Figure 4) provides the narrowest vertical FOV in natural standing head position. Viewers must look up to maximize their sense of immersion. Slightly reclining seats (4b) provide viewers with an improved FOV. Note that the front seats would ideally be reclined at a greater angle to maximize immersion. Tilting the dome as shown in (4c) improves immersion, especially for viewers towards the front of the theater. Increasing dome tilt and adding stadium seating as shown in (4d) maximizes vertical FOV and is an optimal design for immersive theater where it is desired to deliver a collective audience experience. Optimal immersion is also attained when floating or reclining in a level dome as shown in (4e), or when rows are vertically stacked in a 90-degree tilted dome as shown in (4f).
Figure 4. Unidirectional dome designs including a) standing, b) seated, c) seated with dome tilt, d) stadium seating with dome tilt, e) floating or reclined viewing and f) vertical seating
Floating and vertical designs provide greater isolation from other audience members which is not desirable in a shared theater environment but is ideal for special applications such as digital wellness environment (floating design), flying theater or theme park dark ride (vertical design).
Note that tilting the dome towards the front of the theater allows the projection of foreground under the average viewer’s virtual “gravity horizon” line (Figure 5). In a level dome the spring line is the same as the horizon line. As the dome is tilted, foreground imagery under the horizon is revealed. In practice this virtual horizon can actually be placed anywhere to achieve a desired aesthetic and does not need to follow the audience’s gravity horizon, although the experience is the least disorienting when the two coincide. Since most of the audience is seated behind dome center, the “design eyepoint” can be lifted into the center of the seating deck, thereby lifting the horizon higher up onto the dome.
Figure 5. Variations in dome tilt and formation of a virtual “gravity horizon”
球幕仰角 Dome Elevation Angle
Another variable in screen design is the vertical elevation of the dome with respect to dome center (sometimes called “sweep” or “aperture angle”). Most planetarium domes are perfect hemispheres (180-degree vertical elevation) for legacy reasons. There are, however, advantages to constructing domes that are less than (6a) a hemisphere (Figure 6). Hypohemispheric dome screens have less surface area and thus tend to cost less, are easier to project onto using multiple projectors, and result in improved contrast as we shall see. The 165-degree dome (6b) was originally chosen by IMAX Corp. since it dropped the spherical center closer to the audience’s eye plane which minimizes geometric distortion as viewers move off-axis.
Figure 6. Variations in dome vertical elevation or aperture angle
弹簧线高度 Springline Height
Finally, we look at image field-of-view and placement of audiences within immersive spaces and placement of the springline. It is certainly possible to completely surround a viewer with complete sphere with no springline at all. Full-sphere theaters have been built for research such as the AlloSphere (KucheraMorin, et al., 2014), and have found applications in exhibitions and museums (Maceda, 2015; GOTO, 2016; Proctor, 2017).
However, the presence of a fully populated audience—whether seated or standing—limits a typical viewer’s unobstructed field of view to approximately a hemisphere. That is because, in a densely populated space, there are audience members sitting or standing in front of, behind, and to the sides of us which obscures our field of view. So, while we could extend the dome screen to the floor level or below, sightlines to these pixels—which are costly—are blocked for most audience members and do not appreciably add to the audience experience. Note that this primarily applies to performing arts theaters, concert venues and other densely populated venues.
(Figure 7) shows the lowest sight lines for a seated audience terminate slightly above the heads of the audience (7a)—often referred to as the “eye plane.” This suggests placement of the springline no lower than what is visible for a majority of the audience.
Figure 7. Sightline criteria for placement of springline and stage
A tilted dome demands a tilted eye-plane which is satisfied by stadium seating as shown in (7b). And when a stage is placed in the theater, if it is desirable for the audience to see the surface of the stage, then the front surface of the stage should not be much lower than the lowest visible springline as shown in (7c).
Most dome theaters utilize one or more projectors illuminating a perforated aluminum or fabric dome screen. The projection system must seamlessly map a 1:1 aspect ratio Dome Master frame over the entire surface of the dome. Various projection display configurations are outlined below.
Single-projector systems utilize ultra-wide angle “fisheye” lenses to illuminate all or most of the dome screen. While high-brightness single-projector systems have been introduced for large domes including IMAX ® Dome, the limited resolution and brightness of a single projector can be an issue. Most high-resolution projectors have a 1.6:1, 1.78 or 1.9:1 aspect ratio which does not match the 1:1 fisheye fulldome format. This either requires vertically truncating the projection in the back of the theater or limiting the full fisheye frame to the vertical resolution (Figure 8).
Figure 8. Fisheye projection options for single projector with 1.9:1 aspect ratio
In general, most single-projector systems are found in smaller domes up to 10-12m in diameter and require 4K or 8K pixel resolution projectors for best results (Figure 9). The projector is ideally placed in the center of the dome (9a) but may be placed towards the front or rear of the dome with appropriate digital remapping and intensity gradient compensation. Single projector systems are simpler to setup and align and are therefore much better suited to portable dome applications. Dual fisheye configurations are also possible that use a second projector to fill in the truncated area as shown in (9b). A single edge-blend between two projectors is accomplished using geometric warping and soft-edge masking. Navitar (2019) publishes an excellent tutorial on single and dual projector dome systems.
Figure 9. Single, dual & six-projector edge-blended projector layout
多投影机边缘融合系统 Multi-Projector Edge-Blended Systems
Larger systems typically require multiple projectors to allow scaling of brightness and resolution. The majority of larger domes use multiple video projectors—up to two dozen or more in large-scale domes or specialized simulator applications. The projectors must sufficiently overlap to allow geometric alignment and soft-edge-blending to create a single seamless image covering the entire surface of the dome screen. Auto-alignment systems use one or more cameras in conjunction with structured light grids to compute u/v warping and blending maps to create a single seamless image from multiple overlapped projectors. Projectors are typically mounted around the periphery of the dome, projecting across the dome diameter (9c).
Note that the projector black level in blend regions is additive which introduces a distracting hard-edged fixed pattern in the black level which effectively reduces the sequential (on/off) system contrast. There are two solutions to restore black level uniformity. The first is to simply raise the black level outside of the blend regions—a simple solution for auto-edge-blend systems. This reduces black-level contrast to a lesser—but uniform—value across the frame. This approach works particularly well with projectors that have high sequential contrast (ideally without the use of dynamic contrast). But most large-venue projectors are limited to 2000:1 to 5000:1 sequential contrast a best, resulting in discernable loss of contrast. The second solution applies a physical optical mask somewhere in the optical train (outside of the projection lens in some systems) to create physical blends that function at all light levels. This is not an ideal solution however it can work very effectively. Emerging HDR (high-dynamic range) projectors promise to alleviate the need for masks or black-level boosting altogether.
There are numerous choices for video projectors and lenses. The most popular choices of projection technology are single-chip or three-chip DLP (Texas Instrument’s Digital Light Processor), LCOS (liquid crystal on silicon) or organic LCD (liquid crystal display) spatial light modulators (SLMs), with DLP technology clearly leading in high-end projectors. Projectors with up to 4K horizontal pixels have been manufactured with some 8K projectors entering the market. Lower cost versions use “wobulators” to optically displace pixels by a fraction of a pixel into several positions per frame—each position a fraction of a pixel—to attain higher virtual frame rates.
Projector lenses are rated according to throw ratio. The throw ratio specifies the ratio between image width and throw distance. A 1.5:1 throw ratio, for instance, will produce an image one unit wide when the projector is placed 1.5 units of distance from the screen. Ultra-short throw lenses are now available with throw ratios of 0.3:1 or less. Precautions must be taken, however, to assure that the lens will properly focus across the curvature of the dome. It is rare that projector manufacturers publish this data—empirical testing is a must.
Sequential contrast is another parameter that must now be tested empirically. In recent years manufacturers have universally abandoned the ANSI intra-frame (checkerboard) contrast specification in favor of dynamic contrast specification. Dynamic contrast uses variable iris to throttle down projector brightness in low light scenes, achieving dynamic contrast ratios of 20,000:1 or more. Dynamic contrast cannot be used on multi-projector systems unless the dynamic parameters of all projectors are synchronized. And, because of the very wide field of view, dome imagery does not lend itself to dynamic contrast over the entire display. Most projectors allow dynamic contrast to be disabled, and some allow detailed control over parameters or synchronization of multiple projectors. Without dynamic contrast activated some projectors are left with sequential contrast of 500:1 or worse, so empirical evaluation of contrast is recommended.
Stereoscopic 3D is also possible in digital domes; however, it is difficult to fabricate large polarizationpreserving projection screens with sufficient precision. Instead, active sequential LCD shutter glasses or optical wavelength multiplex imaging technology such as INFITEC ® (used by Dolby and others) is typically required for eye separation. There are also geometric compromises in 3D stereo imagery in domes with a large off-axis viewing volume that make it more difficult for audiences to fuse stereo images from obtuse viewing angles.
立体 3D 技术在数字球幕影院的应用是可行的，然而，制作大型且精度足够的偏振投影屏幕存在挑战。相反，通常因此，通常采用主动顺序液晶快门眼镜或光学波长复用成像技术，例如 INFITEC ®（被 Dolby 等公司采用），来实现左右眼的图像分离。在具有大偏轴观看体积的球幕中，立体 3D 图像可能会面临几何上的妥协，这使得观众从钝角视角融合立体图像变得更加困难。
图形服务器 Graphics Server
A basic multiprojector digital dome system is shown in (Figure 10). One or more central media servers takes dome master frames, breaks them into sub-frames with proper warping and feeds the sub-frames to multiple video projectors. Image source can either be realtime 3D rendered visuals or pre-rendered video streaming from SSD drives in high-speed RAID configuration. An outboard digital audio processor takes synchronized audio stems or multiple audio tracks in virtually any surround or 3D audio format and remaps them into the theater’s speaker array. A separate audio channel is provided for vibroacoustic transducers on the chairs, seating deck or floor.
图 10 展示了一个基础的多投影机数字球幕系统配置。中央媒体服务器接收球幕母版帧，将其分解成经过适当变形处理的子帧，并将这些子帧发送到多个视频投影机。图像源可以是实时 3D 渲染的视觉效果，或者是从高速 RAID 配置的 SSD 驱动器中流式传输的预渲染视频。外部数字音频处理器接收同步音频分轨组（stems）或多个音频轨道，并将其重新映射到剧院的扬声器阵列中，同时为椅子、座位甲板或地板上的振动声学换能器提供单独的音频通道。
The system includes an auto-alignment camera or array of cameras to capture projected structured light allowing the display to be geometrically aligned and automatically edge-blended using one of several commercial algorithms. Another array of cameras allows tracking of performers on the stage. The server plays video clips and 3D interactive scenes according to a cue list or timeline.
该系统包括一个自动对齐相机或相机阵列，用于捕捉投影的结构光，实现几何对齐，并使用商业算法自动进行边缘混合。另一个相机阵列用于跟踪舞台上的表演者，服务器根据提示列表（cue list）或时间线（timeline）播放视频剪辑和 3D 交互场景。
Multiple auxiliary video inputs fed by a video switcher allowing external video sources to access the display. The server accepts any planar or 360 format (rectangular, equirectangular 360, equipolar dome masters, cubic etc.) and remaps it into dome coordinates with low latency using 2D or 3D mapping. External inputs may be layered or mixed with internal realtime 3D or pre-rendered 360 video. In this manner the dome server becomes a dedicated remapping server allowing multiple outboard image generators including stage mapping computer, audience interactive computer, digital lighting server, GPU supercomputer, multiple VJ servers, live immersive camera feeds and other third-party sources to be seamlessly orchestrated to deliver powerful audience experiences.
视频切换器提供多个辅助视频输入，允许外部视频源接入显示系统。服务器能够接收任何平面或 360 度格式的输入（如矩形、等距圆柱 360 度投影、等极球幕母版、立方体等），并使用 2D 或 3D 映射技术以低延迟将其重新映射到球幕坐标。外部输入可以与内部的实时 3D 或预渲染 360 视频层叠或混合。这样，球幕服务器充当了一个专用的重映射服务器，使得多个外部图像生成器（包括舞台映射计算机、观众交互计算机、数字照明服务器、GPU 超级计算机、多个 VJ 服务器、实时沉浸式摄影机流和其他第三方源）能够无缝协同工作，共同创造出色的观众体验。
A show control interface allows coordination with lighting, motion control rigging, stage props, musical instruments, audio mixers, sensors and other peripheral devices through SMPTE timecode, MIDI, DMX and related control interfaces.
Figure 10. Basic multiprojector display system for live-performance dome theater
图像亮度 Image Brightness
With their large surface areas, digital dome projection screens are hungry for light. In cinema systems image “brightness” or luminance is often measured in US units of foot-Lamberts (fL). The corresponding SI units for luminance are candela/m2 and are often referred to as nits (1.0 fL = 3.426 candela/m2). The SMPTE standard ST 431-1: 2006 for digital cinema requires screen luminance of 14 fL or 48 nits for full white (SMPTE, 2006). Projection domes, whether film or digital, rarely achieve these brightness levels. The Giant Screen Cinema Association’s DIGSS 2.0 requirements for giant screen theaters requires at least 3 fL for full white with an aspirational goal of 6 fL (DIGSS, 2018). Table 1 lists required projector luminous flux (in lumens) to achieve various luminance values (in both fL and nits) for a range of dome diameters.
Table 1. Projector luminous flux (in lumens) versus screen luminance (fL) for various dome diameters. Assumes screen reflectance of 0.2, dome elevation 165 degrees and projector overlap factor 0.8
For live performance with stage lights and other ambient lighting it is highly recommended to have a minimum luminance of 4 fL. While higher luminance is always desirable, it is often not economically feasible. For instance, achieving 4 fL in a relatively small 15m dome (seating capacity 120) can be accomplished using three 30k lumen projectors. However, achieving the same light level in a 60m dome (seating capacity 2500) requires 44 ea. of those same projectors. Even using the brightest commercially available projector (75,000 lumens) still requires 18 ea. projectors. It is easy to see the temptation to economize by a) reducing the number of projectors and decreasing image luminance, b) reducing the number of projectors and using a higher screen reflectance to maintain the same image luminance (which sacrifices contrast as we shall see) or c) reducing the vertical elevation (aperture angle) of the screen thereby allowing a reduction in the number of projectors without sacrificing either luminance or contrast (although there may be loss in vertical resolution depending on projection geometry).
Note that projectors can be double-stacked and aligned using internal warping, thereby halving the required number of unique server channels. Double-stacking is common in the industry and most projectionists can quickly manually align projector stacks.
Image resolution is another critical measure when designing dome displays (or any display, for that matter). Note that, since the introduction of digital projection, display application engineers now typically express resolution in resolvable pixels (pixel resolution) rather than the optical engineering definition of resolution which is based on resolvable line-pairs. Resolution in digital domes is expressed in pixels per degree or—quite often—in the number of pixels per 180 degrees. This is because the standard image map for domes—known as the Dome Master—is an equidistant polar (fisheye) frame with vertical and horizontal axes both representing 180 degrees (from springline to springline). The same 180-degree dome master frame is mapped to the entire dome, regardless of dome elevation (aperture) or dome tilt. Digital domes have an interesting property. If one stands at dome center, domes with the same number of pixels will appear to have the same pixel resolution regardless of scale. Resolution is measured by pixels per degree, a factor that remains constant with dome diameter. Of course, the pixel size itself does scale with dome diameter, since the number of pixels around the circumference remains fixed as the circumference increases. If we design theaters with seating or standing areas measured in fractions of dome radius, then our pixel resolution design criteria will also scale with dome diameter.
Figure 11. Eye-limited resolution as a driver for pixel resolution at various viewing locations expressed as a fraction of dome radius (scales with dome size), with resultant pixel width for various diameters
The acuity of the human eye with 20/20 vision is approximately one arcminute. Dome resolutions resulting from requirement of eye-limited resolution at four different viewing locations (Figure 11). These viewing locations (front seat, dome center, rear center and rear seat) are expressed in units of dome radius, therefore the resolution requirements are scale-invariant with respect to dome diameter.
Large-scale digital planetariums and giant screen domes typically employ an industry standard of 4K x 4K dome master resolution with a small number of 8K x 8K systems deployed. As shown, a resolution of 5.4K is required to match the resolution of the human eye from the rear-most seat. An 8K dome display falls below eye-limited resolution just behind dome center. And if eye-limited resolution is desired at the front row—assumed to be 0.5 radii in front of dome center—a 22k dome master is required.
大型数字天文馆和巨幕球幕普遍采用行业标准的 4K x 4K 分辨率作为球幕母版，同时也部署了少量 8K x 8K 系统。如图所示，为了与最远座位观众的人眼分辨率相匹配，需要的分辨率是 5.4K。而在球幕中心后方，一个 8K 球幕显示器的分辨率略低于视觉极限分辨率。如果想要在前排——假设在球幕中心前方 0.5 半径的位置——达到视觉极限分辨率，那么所需的球幕母版分辨率将是 22K。
This raises the question of the need for eye-limited resolution. Projection systems have evolved to have very high pixel fill factors—up to 92% which minimizes the “screen door” effect when viewing the projected image at closer distance than the eye-limited resolution limit. From experience, the eye is very tolerant to a factor of two (or more) drop in pixel resolution below eye-limited resolution. So an 8K dome master will likely be sufficient into the foreseeable future. Such is not the case with LED displays which have a very low fill factor, allowing the eye to easily discern individual pixels when under the eye-limited resolution threshold. So, even in the case where 8K content is being projected on an LED screen, the pixel resolution will want to be near eye-limited resolution to prevent screen-door artifacts.
随着投影系统的发展，它们已经能够实现非常高的像素填充因子，最高可达 92%。这一进步在接近视觉极限分辨率的距离观看投影图像时，显著减少了“屏门”效应的影响。实际上，人眼对于低于视觉极限分辨率两个（或更多）数量级的像素分辨率下降表现出了很高的容忍度。因此，一个 8K 球幕母版在可预见的未来应该已经足够满足需求。然而，对于 LED 显示器来说，情况就有所不同了。它们的像素填充因子相对较低，当分辨率低于人眼的分辨率阈值时，人眼可以轻易地分辨出单个像素。因此，即使在 LED 屏幕上播放 8K 内容，也期望像素分辨率尽可能接近视觉极限分辨率，以防止出现屏门伪影，确保观众获得最佳的观看体验。
交叉反射对比度 Crossbounce Contrast
An often-overlooked factor in dome theater design is the effect of scattered light or “crossbounce” (also called inter-reflection)—that is, light from projected images that scatters back onto the dome instead of into the eyes of the audience causing loss of contrast. This is one of the most common, difficult to solve and often most misunderstood limitations of projection dome image quality. Unlike standard cinema screens that are substantially flat, dome screens are concave, allowing them to reflect not only projected light which forms the primary image, but light from the primary image that scatters back onto the dome itself. The effect is a reduction of image contrast due to an increase in ambient light that limits the blackest black that can be obtained.
Interestingly, crossbounce contrast is highly dependent upon the nature of the projected dome imagery. Bright images such as white clouds – as one might expect when looking up at a daytime sky—can substantially raise the black level over the entire dome screen. Should such an image also contain dark areas, such as looking into the mouth of a cave or dark shadows under trees or rocks, these dark areas of the image will appear quite washed out. On the other hand, sparse imagery against black—such as a starry night sky—produces very little scattered light. In this case contrast is likely limited by the projector’s sequential contrast (on/off black level) or ANSI contrast (typically limited by scatter in lens and optics) and not contrast loss due to crossbounce.
In (Figure 12) we illustrate crossbounce by tracing a single ray of light with illuminance L. This ray strikes a grey Lambertian screen (which scatters equally in all directions) with reflectance r. The resulting image illuminance seen by an observer is therefore proportional to r. Crossbounce light is attenuated a second time relative to the primary reflected image. Therefore, the intensity of a crossbounce ray varies proportional to r2 as r is varied. The overall crossbounce contrast ratio (r / r2) varies approximately as 1/r. So as the reflectance is lowered, crossbounce contrast, which is inversely proportional, increases.
在图 12 中，我们通过追踪具有照度 L 的单束光线来说明交叉反射现象。这束光线击中了一个具有反射率 r 的灰色朗伯体屏幕（该屏幕在所有方向上均匀散射光线）。因此，观察者看到的图像照度与 r 成正比。交叉反射光相对于主要反射图像再次减弱。随着 r 的变化，交叉反射光的强度与 r ²成比例变化。整体交叉反射对比度比率（r / r ²）大约与 1/r 成反比。因此，随着反射率的降低，与之成反比的交叉反射对比度会增加。
Figure 12. Crossbounce of incident light ray with luminance (L) onto screen with reflectance (r)
This simple model neglects image shape and intensity variation and more complex specular properties of dome screens. The real-world factors contributing to crossbounce on arbitrary images are complex and can be solved through numeric modeling (Hazleton, 2016).
In practice a standard method for measuring the crossbounce contrast ratio is to project a black and white checkerboard over the entire dome surface and measure the ratio of luminance between the white squares and the black squares (Lantz, 2004).
Checkerboard contrast can be derived using integrating sphere theory (Ganter, 2012) and is given by:
C = 2 (1 + A/2) / A
Where A is the amplification factor given by A = 1 / (1 – r/2 * (1-cos(Ø/2))− 1
Where r is the dome screen reflectance (also called gain) and Ø is the vertical dome elevation.
棋盘格对比度可以用积分球理论（Ganter，2012）推导出来，如下： 其中， 是放大系数，公式为： Checkerboard contrast as a function of dome gain and vertical elevation is shown in Table 2. Many traditional planetariums utilize hemispheric screens (elevation of 180 degrees) with a gain of 0.65 which works well for star projection where the imagery is mostly black with pinpoint stars and planets. However, this results in a checkerboard contrast of less than 4:1 which is concerning for cinematic content creators as it severely limits image quality for brighter content. In comparison, digital cinema systems are generally expected to maintain checkerboard contrast of 100:1 or more.
Table 2. Checkerboard contrast vs dome gain and vertical dome elevation
Lowering screen gain and decreasing elevation angle can drastically improve the crossbounce contrast ratio. Halving the screen reflectance approximately doubles checkerboard contrast. IMAX ® Dome screens designed for cinema have typical reflectance of 0.28 or less. The Vulcan Holodome, a spherical theater designed by Vortex Immersion Media, Inc., used a screen with reflectance of 0.13. This dark grey screen decreased image brightness nearly by a factor of eight requiring much brighter projectors to compensate for the optical loss. Unfortunately, many existing digital domes were originally constructed for star projection with a screen reflectance exceeding 0.6 which is entirely unsuitable for cinematic content. Even the best projection dome screens would be hard pressed to achieve 100:1 checkerboard contrast.
A dome with elevation angel of 155 degrees would need to have a reflectance of 0.025 to achieve 100:1 contrast which would attenuate projected light by a factor of 40. Clearly, if digital domes are to achieve their full potential to faithfully reproduce real or simulated imagery, another solution must ultimately be found.
Another method for increasing crossbounce contrast is to increase screen gain. Basically, a high-gain screen concentrates reflected light into the specular direction rather than scattering light equally in all directions. If the projector is a single-lens design placed near dome center, a high-gain dome screen will concentrate reflected light back down towards the projector—and thus, concentrate reflected light into the audience seating area. High gain dome screens have been manufactured for IMAX ® Ridefilm and specialty simulator systems. However, while this approach works for smaller single-projector systems it does not scale up to multi-projector edge-blended systems in theaters since image brightness of overlapped projectors in blend regions varies according to viewer position within the theater. Simulator systems have a single viewer with a small “viewing volume” and therefore can compensate for viewer position.
Larger dome theaters require multiple projectors to scale up brightness and resolution. Novel approaches to compensating for crossbounce have been summarized by R öß ner, Christensen, and Ganter (2016) and include real-time computational “reverse radiometry” (Bimber, Grundhofer, Zeidler, Danch & Kapakos, 2006), dome screens with microstructures to provide selective gain (Bublitz, D. 2011), camera based approaches (Habe 2007) or photochromic screens that can selectively subtract light (Takeda, 2016). These approaches have not been applied to large-scale digital domes as they are either highly computationally intensive, expensive to manufacture or not suitable for large-scale dome theaters.
R öß ner、Christensen 和 Ganter（2016）总结了补偿交叉反射的新方法，包括实时计算“逆辐射度学”（Bimber, Grundhofer, Zeidler, Danch & Kapakos, 2006），采用微结构以提供选择性增益的球幕屏幕（Bublitz, D. 2011），基于相机的方法（Habe 2007），或可以有选择地减光的光致变色屏幕（Takeda, 2016）。这些方法尚未应用于大型数字球幕，因为它们要么计算强度高，要么制造成本高，要么不适合大型球幕剧院。
The ultimate digital dome display will likely require a departure away from projected display technology altogether.
最终，数字球幕显示器可能需要完全脱离投影显示技术。
LED 显示系统 LED Display Systems
The use of discrete LED panels has the potential advantage of greatly reducing cross-dome scatter, opening the door to high-dynamic range (HDR) dome theaters (Kleiman, 2019; Campos, 2019; Brennesholtz, 2019). LED’s are also brighter, easily exceeding the 55 nits requirement for digital cinema resulting in better perceived color saturation. Recent innovations in LED domes include perforated panels for sound transmissivity and custom fabricated panels for spatial pixel uniformity over the spherical surface.
使用独立式 LED 面板有潜力显著降低球幕内的散射现象，这一技术进步为高动态范围（HDR）球幕影院的实现铺平了道路（Kleiman, 2019; Campos, 2019; Brennesholtz, 2019）。LED 光源的亮度更高，轻松达到数字电影所要求的 55 尼特以上亮度标准，进而提升色彩饱和度的感知效果。近期在 LED 球幕技术上的创新包括：开发了带有穿孔设计以实现声音传输的面板，以及用于确保球形表面空间像素分布均匀的定制面板。
The primary barrier to entry for LED domes is cost, which is bound to come down as LED technology advances. LED screens and other emissive surfaces (OLED, printed polymers) are likely the future of this format. Dome theaters should be designed to be future-proof and accommodate potential upgrade to emissive screens.
LED 球幕技术的普及面临的主要障碍在于其成本。随着 LED 技术的不断进步，预计这一成本将会逐渐降低。LED 屏幕以及其他自发光显示技术（如 OLED、印刷聚合物）有望成为球幕影院未来的发展趋势。因此，球幕影院的设计应当具备前瞻性，以便未来能够适应可能的自发光屏幕技术升级。
As shown in Figure 10, a 60m diameter 22k pixel dome requires 4.4mm pixel pitch to be eye-limited resolution—well within the capabilities of current discrete LED panel design.
Projection mapping is a powerful tool to transform any spatial environment where ambient light levels can be controlled. Textures can be projected onto walls, floors, furniture, and more. In the case of a live stage performance, large digital sets can be computer cut or 3D printed to exactly resemble objects projected within the immersive dome scene.
投影映射技术是一种强大的工具，能够对任何可以控制环境光线水平的空间进行转换。这项技术允许我们将纹理投影到墙壁、地板、家具等表面上。在实时的舞台表演中，大型的数字布景可以通过计算机切割或 3D 打印技术制作出来，这与在沉浸式球幕场景中投影的对象有着异曲同工之妙。
The “holy grail” of live performance immersive design is where a vast spatial scene on the dome comes seamlessly into the room and onto an LED or projection-mapped stage. The effect is working when the eye cannot tell where the dome ends and the stage begins. This effectively opens a portal bridging the virtual cyber world and the real physical world (Zhang, Shen, Zhang, Zhu & Ma, 2019).
当球幕上展示的宏大空间场景无缝地延伸至房间内，并投影到使用 LED 或投影映射技术的舞台时，现场表演沉浸式设计的“圣杯”就实现了。当观众无法区分球幕的边界与舞台的起点时，这种效果便得以完美呈现。这不仅创造了一种视觉上的奇迹，也象征着虚拟网络世界与现实物理世界之间的界限被打破（Zhang, Shen, Zhang, Zhu & Ma, 2019）。
These two worlds become seamlessly stitched together when, for instance, a sun (projected onto the dome) arcs across the sky and the shadows of stage props (projected onto the stage) shift position, tracking the sun. A performer then lifts a glowing orb and the light rays illuminate 3D objects in the cyber world (projected onto the dome), seemingly emanating from the orb which is being tracked across the stage. The performer then walks on a treadmill while the 3D projected imagery on the dome scrolls past. Another powerful effect is the tracking of performers and use of video projectors as stage lights. By placing light onto the performers only we can avoid spillage of ambient stage lighting onto projected scenes. We can also project textures onto moving performers.
例如，当球幕上投影的太阳仿佛在天空中移动，而投影到舞台上的道具阴影也随之改变位置时，这两个世界便实现了无缝的融合。表演者随后举起一个发光的球体，其光线照亮了投影在球幕上的 3D 物体，而这些光线似乎正是来自于舞台上实际追踪的球体。接着，表演者在跑步机上行走，而球幕上的 3D 投影图像则随之滚动，创造出一种动态的视觉效果。此外，将视频投影技术用作舞台灯光也是一种强大的效果。通过仅将光线投射到表演者身上，我们可以避免环境舞台照明对投影场景的干扰。我们甚至可以将纹理投影到移动的表演者身上，进一步增强表演的沉浸感和视觉冲击力。
音频系统 Audio Systems
There are numerous surround sound configurations utilized in dome theaters, from 5.1 surround, 7.1 and 11.1 to custom 23.2 systems and beyond (Gaston, 2008). Three-dimensional audio processing is also coming into vogue. Standard commercial systems may conform to THX, be modeled after open source formats such as ambisonics or proprietary formats such as Dolby Atmos, Auro 3D and DTS. It is highly desirable to place audio in spatial configurations to match immersive visuals. More sophisticated wavefield synthesis techniques such as IOSONO and Auro 3D Max offer holographic sound reproduction allowing virtual audio sources to appear anywhere within the dome space.
The primary goal of a modern digital dome audio system should be the ability to accept nearly any set of audio stems with directional information or any set of surround channels and re-map the audio to the venue’s speaker configuration. Vibroacoustic transducers on the floor or chairs can also transmit low frequency vibration through the sense of touch. This is best provided by a separately mixed channel rather than deriving it from the sub bass channel.
Two primary theater types are recommended for standardization as arts and entertainment venues: Multipurpose domes and CineTheater ™ domes.
作为艺术和娱乐场所标准化，推荐两种主要的剧院类型：多功能球幕和 CineTheater ™球幕。
多功能球幕 Multipurpose Domes
The first and most popular design for mobile domes is a simple non-tilted dome with level floor and removable seats. This architecture is a multipurpose design that serves as a digitally themed ballroom or event center and is suitable for banquets, dance parties, concerts and simple theatrical productions (Figure 13). Provisions must be made for storage of chairs, tables bars and other equipment that may be swapped in and out according to event type.
The optimal design for shared theatrical productions is the CineTheater ™ with tilted springline, stadium seating deck and projection mapped (or LED) stage (Figure 14). Such a theater can maximize the illusion of a seamless merging of real and virtual worlds.
There are substantial challenges to the seamless integration of video backdrops and set pieces in dome theaters since there is no theatrical proscenium to hide lights or drop scrims, set backdrops, props and rigging. These effects must instead be raised from underneath the stage or dropped from overhead through moving ports in the dome screen. A number of innovations in theatrical staging are now being developed and standardized allowing productions to be easily distributed to a network of CineTheaters ™.
Digital domes are general purpose immersive environment that are extremely flexible and can be used in a variety of ways to deliver sit-down, stand-up and walk-through experiences.
数字球幕是一种通用的沉浸式环境，非常灵活，可以用多种方式提供坐下、站立和步行穿越的体验。
360 度电影体验 360 Cinema Experiences
Domes are best known for delivering amazing 360 cinematic journeys, from IMAX films to thrill rides and visual music entertainment. Like VR, 360 cinemas can evoke deep empathy and create breathtaking spectacles.
现场 CineTheater ™表演 Live CineTheater ™ Performances
Live performers can step into and interact with projected immersive environments which can serve as digital sets. CineTheaters ™ have features that support live performances including projection mapped stage and props, video-based lighting, talent facilities, and more.
Broadway, Cirque and large-scale theatrical stage shows rely on extensive physical sets, props, and expensive stages to create massive spectacles for attendees. As discussed by Passy (2013), mounting a Broadway show typically costs 25 million. At the high end, for the Cirque du Soleil show KA they built a custom 30 million to produce (Fink, 2004).
CineTheaters ™ require a one-time investment in AV equipment which—similar to cinema— utilize live-action cinematography or visual effects to create the illusion of nearly any scene imaginable. But instead of viewing the scene within a rectangular frame on a wall, the digital dome wraps the environment around audiences to evoke a strong sense-of-presence. Live characters can enter the virtual environment and command it, effectively commanding the nervous systems of the audience.
While visual effects can be expensive, once produced, a CineTheater ™ show can be rapidly and inexpensively mounted from a hard drive along with a handful of simple projection-mapped set pieces and minimal rigging. The cast is reduced because some of the performers can be rendered into pixels instead of being physically present for each performance.
A wide variety of events can be enhanced using 360 environmental backdrops. Cocktail parties, banquets, and gatherings of all kinds can benefit from abstract immersive scenes such as starry night sky, beaches, palaces, cityscapes, popular art and more. Scenes can be static or slowly changing with looping visuals (i.e. moving clouds, trees blowing in wind, birds soaring overhead). This is sometimes referred to as “digital wallpaper.”
The dome can serve as a group immersive portal into virtual worlds including interactive games, esports, and metaverse-based events. Walk-through immersive environments can be created that include projection mapping gesture-based textures onto real-world sets and objects.
Massively multi-user online virtual environments and games (MMO’s) such as Second Life, Fortnite and World of Warcraft have attracted millions of users. Virtual world participants don avatar personalities and navigate through 3D shopping malls, nightclubs, games, and other user-generated virtual worlds. Avatars can work, eat, sleep, meet one another, make love, get married, raise children, purchase real estate, attend game shows, change gender, and act out fantasies. Users of MMO’s build personalities, skills and attributes over time, just as in the real world.
大型多人在线虚拟环境和游戏（MMOs），如《模拟人生》《堡垒之夜》《魔兽世界》，已经吸引了数百万用户。在这些虚拟世界中，参与者扮演虚拟角色，他们在 3D 购物中心、夜总会、游戏和其他用户生成的虚拟空间中导航。虚拟角色可以体验到工作、吃饭、睡觉、社交、恋爱、结婚、抚养孩子、购买房地产、参与游戏节目、改变性别和实现各种幻想。随着时间的推移，MMO 的用户会在游戏中建立自己的个性、技能和属性，就如同在现实世界中一样。
Virtual worlds including user generated environments created in MMO’s can ported into physical worlds as walk-through SAR environments using 3D-printed objects from the virtual world.
包括 MMO 中用户生成的环境在内的虚拟世界，可以通过使用虚拟世界中的 3D 打印物体，作为步行穿越的空间增强现实（SAR）环境导入到物理世界中。
讲座和演示 Lectures and Presentations
Slide-show presentations cannot compete with the power of immersive intelligent spaces. Panelists and presenters can teleport audiences into virtually any place on Earth or beyond, including rare ecosystems, astronomical datasets, microscopic domains, or virtual worlds. Executive presentations can command vast amount of information and visual simulations. Self-help speakers can add dramatizations and experiential elements.
Performers or dancers can create live interactive art on the dome and audiences can interact with dome content with user interfaces such as mobile phones, wands, game controllers, motion-captured gestures, or speech. The resulting “digital Cirque” performances can simulate massive theatrical spectacles that would otherwise be costly to fabricate. Virtual characters and sets can transcend reality as they are not bound by physical limitation. AI-driven “synthespians” can access personal data to personalize the audience experience for those who opt in.
表演者或舞者可以在球幕上创作现场互动艺术，观众可以通过各种用户界面，如移动电话、魔杖、游戏控制器、动作捕捉手势或语音，与球幕内容进行互动。这种“数字马戏团”式的表演能够模拟大规模的戏剧奇观，而成本却远低于传统方式。虚拟角色和布景能够超越现实世界的物理限制，而 AI 驱动的“合成演员”甚至可以根据个人数据为观众提供个性化的体验。
现场音乐会和舞蹈音乐活动 Live Concerts and Dance Music Events
Video jockeys (VJ’s) can take over the dome and perform to live music, bringing stand-up concerts and dance music events to an entirely new level. Advanced dome re-mapping servers support real-time frame capture and re-mapping allowing VJ’s to directly drive the dome using their own VJ server and content. Advanced AI image generation allows real-time style transfer and other artistic effects. The powerful nature of 360 visuals elevates the VJ to equal status with the DJ. Gesture-driven “dome jockey” interfaces go further to allow a single performer to command both musical and visual performance from a single console.
Global audiences can be engaged in live events (Figure 15) through 360 multicasting (15a) and domecasting (15b). In these scenarios a 360 camera captures a scene, either within an immersion dome – such as a concert performance—or other live event and streams the event into other domes or headsets. In domecasting, the remote domes can enhance the experience with the addition of a local moderator or “shoutcaster” and additional video streams and information channels which can be displayed around the dome to augment the 360 “God’s eye” view.
To illustrate trends and state-of-the art in the industry here are a handful of real-world case studies across several application areas from pioneering companies making strides in this field.
Big brands want to dazzle audiences with memorable experiences and are willing to pay handsomely for it. Experiential marketing continues to be an on-ramp for newcomers in immersive media and an important proving ground for new tech coming out of the labs. These case studies demonstrate the power of immersive media to deliver unique and compelling brand experiences.
The Nokia Lab immersion dome and digital projections were created by VIM for the 2012 SXSW festival in association with High Beam Events and featured a tri-dome translucent air-supported design with three 12m diameter dome sections. Air-supported structures require revolving doors or air-locks for ingress and egress that maintain the required positive inflation pressure. This project used an unusual projection architecture with a dome server built upon an interactive engine by Derivative called TouchDesigner which provided real-time 3D animation capability. Experiences included a variety of immersive environments created in cooperation with Pixomondo (water, igloo blocks, rotating brand elements, and a “Tron-like” world), slowly rotating real-time 3D models of Nokia’s new phone and real-time simulated northern lights that users could interact with using a touch table. Live moderated Twitter feeds were also projected onto the dome and a VJ performed visuals in real-time with live band. The VJ visuals initially filled the entire dome but induced vertigo in some attendees, so the art was scaled down and projected onto the screen of the 3D phone model which was positioned to appear floating above the band.
诺基亚实验室的沉浸式球幕和数字投影是由 Vortex Immersion Media (VIM) 为 2012 年 SXSW 音乐节与 High Beam Events 合作创建的。该项目特色是一个三球幕半透明气动设计，每个球幕直径达 12 米。为了维持所需的正压充气压力，气动结构需要设置旋转门或气闸。该项目采用了一种非传统的投影架构，球幕服务器建立在 Derivative 公司的 TouchDesigner 交互引擎之上，该引擎提供了实时 3D 动画的能力。体验包括与 Pixomondo 合作创造的多种沉浸式环境（如水域、圆顶小屋、旋转的品牌元素和一个类似“电子世界争霸”的世界），以及缓慢旋转的诺基亚新款手机的实时 3D 模型。用户还可以通过触摸桌与之互动，体验实时模拟的北极光。现场主持的 Twitter feed 也被投影在球幕上，VJ 与现场乐队实时表演视觉效果。最初，VJ 视觉效果填满了整个球幕，但这在一些参与者中引起了眩晕，因此艺术作品的尺寸被缩小，并投影到 3D 手机模型的屏幕上，该模型被放置在乐队上方，给人一种漂浮的视觉效果。
Twitter Feeds
This and other translucent dome projects identified the need to both reduce ambient light inside and outside the structure and increase projector brightness (Figure 16). In (16a), the luminous bar washed out projections from the interior, while (16b) shows how exterior street lights (along with automobile headlights) interfered with the experience inside the dome. Text on the dome also had to be constantly rotated so it appeared correctly to viewers both inside and outside the dome.
The NFL Super Bowl Host Committee commissioned an 18m inflatable “Stratosphere” dome structure in 2012 to serve the nearly one million visitors anticipated at the Super Bowl Village, a temporary entertainment zone in downtown Indianapolis. Microsoft’s Xbox came in and sponsored the dome. 3D animated visuals were projected onto the dome interior and exterior.
2012 年，NFL 超级碗主办委员会委托制作了一个 18 米高的充气“Stratosphere”球幕结构，用于在印第安纳波利斯市中心的超级碗村临时娱乐区服务预计近一百万游客。微软的 Xbox 赞助了这个球幕，3D 动画视觉效果被投影到球幕的内部和外部。
A second Xbox dome venue was commissioned for Super Bowl 2013 (Figure 17). The interior included a Kinect interaction station allowing visitors to throw simulated Mardi Gras beads, levitate a playing field with running football players or spin Xbox brand elements around on the dome. The 15 m opaque vinyl covered negative pressure dome was seamlessly projection-mapped on both the exterior and interior. The exterior featured rotating brand elements and provided visibility from many points along the Mississippi river. Snorkel doors reduced the amount of ambient sunlight entering the dome as audiences entered allowing a more controlled ambient light environment. Lighting on interior signage was also delivered using projection mapping to reduce interior ambient light. A smaller 7m dome was attached to the main structure which housed a photo booth experience as visitors exited the venue. Approximately 10,000 visitors passed through the dome over several days.
2013 年超级碗再次委托制作了第二个 Xbox 球幕场馆（见图 17）。内部包括了一个 Kinect 互动站，让游客可以扔模拟的狂欢节珠子，使一个有球员在跑步的足球场悬浮在空中，或在球幕上旋转 Xbox 品牌元素。这个 15 米高的不透明乙烯基覆盖的负压球幕在内外都进行了无缝的投影映射。外部特色是旋转的品牌元素，在密西西比河沿岸的许多位置都能看到。潜望门（snorkel doors）减少了进入球幕时的环境光量，使观众进入时能够更好地控制环境光线。内部标识的照明也使用投影映射来减少内部环境光。一个较小的 7 米球幕连接在主体结构上，作为游客离开场馆时的拍照体验区。大约有 10,000 名游客在几天内进入球幕内部体验。
In 2012 VIM worked with EMC brand managers to craft a story to fly audiences through a “dataverse” showing how private and public clouds could be trusted to serve corporations. In 2013 the same team created a Tron-like world that we navigated in a “dataship” while that illustrated tools for IT security and governance. Both pieces included animation with live-action elements and visual effects produced with support from Pixomondo, an Academy award-winning visual effects house that won three Emmys for their dragon animations on HBO’s Game of Thrones (2011-2019) and an Oscar for its work on Martin Scorsese’s Hugo (2011).
在 2012 年，Vortex Immersion Media (VIM) 与 EMC 品牌经理携手合作，打造了一个引人入胜的故事，带领观众飞越一个充满“数据宇宙”的环境，展示了私人和公共云服务如何为公司提供可靠的服务。2013 年，同一个团队再次创造了一个类似电影《Tron》的沉浸式世界，观众在一艘“数据船”上航行，体验了 IT 安全和治理工具的强大功能。这两个作品都包括了动画和实景元素，以及由 Pixomondo 支持的视觉特效。Pixomondo 是一家屡获殊荣的视觉特效公司，因其在 HBO 的《权力的游戏》（2011-2019）中的龙动画赢得了三个艾美奖，并因其在马丁·斯科塞斯的《雨果》（2011）中的卓越工作而获得奥斯卡奖。
康斯坦丁球幕 Constantine Dome
NBC Universal commissioned the creation of an original experience for the launch of Constantine (2014), a TV series based on the DC Comics character and 2005 film starring Keanu Reeves. The 3.5 minute animated themed experience was produced in six weeks and included a flythrough of a creepy sanatorium and collision with a ghost train while combating demons. Synchronized 4D effects included wind and strobes. The show was screened 1,200 times over five days.
到了 2014 年，NBC 环球委托 VIM 为基于 DC 漫画角色和 2005 年基努·里维斯主演电影改编的电视系列剧《康斯坦丁》创造一个原创体验。这个 3.5 分钟的动画主题体验在短短六周内制作完成，让观众体验了一次飞越恐怖疗养院的旅程，并在与鬼火车相撞的同时与恶魔战斗。为了增强沉浸感，体验中还加入了同步的 4D 效果，如风效和频闪灯。这个节目在五天的活动期间共放映了 1,200 次，展示了 VIM 在紧迫时间表下创造高质量沉浸式体验的能力。
沉浸式餐饮 Immersive Dining
The Stella Artois’ Sensorium immersive pop-up dining experience in Toronto, Canada featured a sense of taste, sight, smell, sound and touch throughout a five-course meal by celebrated chef Richie Farina. Show in in Figure 20, the event sold out an entire month of servings at $125/plate prior to opening.
A number of immersive arts and entertainment projects have been pioneered in digital planetariums over the years (Lantz, 2009). However, only a small number of dedicated arts and entertainment immersion dome venues exist at this time.
The Vortex Dome, established in 2010 at Los Angeles Center Studios in downtown Los Angeles has served as a public showcase and testbed for immersive mixed-media productions including; 360 ballet, live painting, musicals, EDM experiences, concerts, solo performances, 360 cinema and more (Lantz, 2018). The 15m dome accommodates up to 130 seated or 240 standing and continues to serve as an immersive media development, R&D lab, and showcase studio.
Productions at the Vortex Dome include; Migrations (2011) with visual artist Audri Phillips and composer Winter Lazerus, Blue Apple ballet with visuals by Audri Phillips and choreographer Stefan Wenta, BollyDoll by artist and singer Amrita Sen and composer Anthony Marinelli, Deep, Deeper Deepest with visuals by Audri Phillips and space music artist Steve Roach, Refractor Piano performed by Peter Manning Robinson with visuals by Hana Kim and Klaus Hoch (multicast in 360 VR), a performance by the band Braves with visuals by Brianna Amore, and Ceremony and Mesmerica visual music performances with musician, James Hood.
Vortex Dome 的制作涵盖了多种艺术形式，包括与视觉艺术家 Audri Phillips 和作曲家 Winter Lazerus 合作的 Migrations（2011）、由 Audri Phillips 提供视觉效果的 Blue Apple 芭蕾、由艺术家和歌手 Amrita Sen 和作曲家 Anthony Marinelli 创作的 BollyDoll、由 Audri Phillips 提供视觉效果，太空音乐家 Steve Roach 创作的 Deep, Deeper Deepest、由 Peter Manning Robinson 演奏，Hana Kim 和 Klaus Hoch 提供视觉效果的 Refractor Piano（在 360 VR 中多播）、由 Brianna Amore 提供视觉效果的乐队 Braves 的表演，以及与音乐家 James Hood 合作的 Ceremony 和 Mesmerica 视觉音乐表演。
Other productions explored in the Vortex Dome including traditional film screenings, performance art, EDM events, poetry readings, experimental art, sound and light meditations, workshops, panels, symposia, immersive film festivals, immersive dining experiences, a Teletubbies premiere party for Nickelodean and more. Next-generation interactive performances are now in production including Audri Phillip’s Robot Prayers.
Video shoots in The Votex Dome include an MTV pilot for a global multicast dance party, two 60 Minutes episodes, a scene from the TV series Castle, an independent sci-fi film, music videos, documentary interviews with legendary jazz musician Wayne Shorter, with rapper Chuck-D from Public Enemy, and most recently a futuristic set for Jay Z’s music video called Family Fued with Beyonc é. Events are regularly live-streamed in 360 video for viewing in VR headsets.
Vortex Dome 也是视频拍摄的热门地点，包括 MTV 全球多播舞蹈派对的试播、两集《60 分钟》、电视剧《Castle》的一幕、一部独立科幻电影、音乐视频、与传奇爵士音乐家 Wayne Shorter 的纪录片采访、与来自 Public Enemy 的说唱歌手 Chuck-D 的采访，最近则是为 Jay Z 与 Beyonc é的音乐视频 Family Fued 制作的未来主义场景。活动定期以 360 视频直播，供在 VR 头显中观看。
SAT
Montreal’s SAT (Soci é t é des arts technologiques) in CANADA features the Satosph è re, an 18 m projection dome with unique multipurpose design dedicated to artistic creation and visualization events (Husband & Barsalo, 2005). The theater opened in October 2011 and has hosted numerous fine arts performances. SAT is one of the world’s first permanent dedicated arts and entertainment dome theaters. The successful business model includes an innovation lab, restaurant and bar making it a frequented local destination in addition to a successful immersive theater.
位于加拿大蒙特利尔的 SAT（Soci é t é des arts technologiques）拥有一个名为 Satosph è re 的 18 米投影球幕，这是一个专为艺术创作和可视化活动设计的多功能场所（Husband & Barsalo，2005）。自 2011 年 10 月开业以来，该剧院已经举办了众多的美术表演。SAT 是世界上第一个永久性的专门艺术和娱乐球幕剧院。其成功的商业模式融合了一个创新实验室、餐厅和酒吧，使其成为当地人频繁光顾的目的地，同时也是一个成功的沉浸式剧院。
Wisdome
And in late 2018 the Wisdome Immersive Art Park opened in Los Angeles’ Arts District featuring two mobile projection domes and three exhibition domes. The venue is open daily featuring the works of visionary artist Android Jones and holds special events such as tribute bands, sound healing, screenings and more.
2018 年底，Wisdome 沉浸式艺术公园在洛杉矶艺术区开业，设有两个移动投影球幕和三个展览球幕。该场馆每天开放，展示视觉艺术家 Android Jones 的作品，并举办特别活动，如致敬乐队、声音疗愈、放映等。
IAIA Digital Dome
The Digital Dome at IAIA (Institute for American Indian Art) is dedicated to exploring “new applications for creative expression, scientific and technical exploration, and the merging of art and technology.” It features a 7m suspended dome that can rotate from a level dome to 90-degree tilt, a 15.1 surround sound system and up to 55 removable seats. The dome has featured numerous art installations and screenings.
Pharos was a temporary mobile dome show featuring Childish Gambino (aka Donald Glover) located in the Joshua Tree desert. Gambino performed five shows over three days in a 49m inflatable dome theater with a 2500-person capacity. The Microsoft-backed production team led by animators Mikael Gustafsson and Alejandro Crow spent two weeks in The Vortex Dome in downtown LA preparing the show which was VJ’d live from dual Unity servers outputting 4K x 4K and re-mapped onto a 12-projector display at 60 fps. Billboard raved that “…the dome was truly the highlight of the night,” and Hip-Hop DX called it “… the most innovative live show this decade.” The second Pharos dome in New Zealand featured five Unreal servers feeding a single re-mapping server, with one rendering the sides of a hemicube.
The largest dome VIM has delivered was a 60m inflatable structure at the Los Angeles Coliseum produced by The Production Club for a well-known game designer. Over 3,000 people enjoyed the world’s most immersive EDM party with performances by Skrillex, Diplo and DJ Snake.
VIM 交付的最大的球幕是一个 60 米高的充气结构，位于洛杉矶纪念体育场，由 The Production Club 为一位知名的游戏设计师制作。超过 3000 人享受了世界上最沉浸的 EDM 派对，演出嘉宾包括 Skrillex、Diplo 和 DJ Snake。
Mesmerica 360
Mesmerica is an extremely successful cross-platform production including a live performance with James Hood (still running at the Vortex Dome LA), a fulldome film for distribution (currently playing in over 23 planetariums and IMAX ® Laser Domes) and a companion VR experience. Directed by Michael Saul, it features James Hood’s cheerful music set to world-class fulldome art. The show is expressly designed to activate positive brain states and a sense of wellbeing. Contributors include Jonathan A.N. Fisher, PhD., an Assistant Professor and the Director of the Neurosensory Engineering Lab in the Department of Physiology at New York Medical College who provided brain visualizations from Neurodome ® and Mark Subbarao, PhD from the Adler Planetarium in Chicago who provided a galactic zoom and brain sequence renders. Other artists include Brianna Amore, Ken Scott, John Banks and Tatiana Plakhova.
Mesmerica 是一个跨平台制作的杰出成就，它包括一场持续在洛杉矶 Vortex Dome 上演的现场表演，一部 Fulldome 电影（目前在 23 个以上的天文馆和 IMAX ®激光球幕上映），以及一个配套的虚拟现实（VR）体验。该作品由 Michael Saul 执导，结合了 James Hood 的欢快音乐和一流的 Fulldome 艺术作品。这个节目专门设计来激发积极的心态和幸福感。项目的贡献者包括 Jonathan A.N. Fisher 博士，他是纽约医学院生理学系的助理教授兼神经感知工程实验室的主任，他通过 Neurodome ®提供了大脑的可视化内容，以及来自芝加哥阿德勒天文馆的 Mark Subbarao 博士，他贡献了银河缩放和大脑序列的渲染。其他艺术家包括 Brianna Amore、Ken Scott、John Banks 和 Tatiana Plakhova。
其他案例研究 Other Case Studies
保罗·艾伦的全息球幕 Paul Allen’s Holodome
In late 2015, a Request for Proposals (RFP) from Paul Allen’s Vulcan was released seeking proposals for a “Holodeck.” The resulting system used four each 4K laser projectors aimed into a semi-spherical screen driven by full-sphere equirectangular movies from TouchDesigner playback and remapping server and two separate Unity servers. Vulcan introduced the Holodome into the MoPop museum in Seattle in early 2019 [3].
The Universal Sphere opened in April 2019 in the lobby of Philadelphia’s new Comcast Technology Center. The 34’ diameter dome theater features a free 7-minute film called The Power of I about the origin and power of ideas produced by Steven Spielberg in cooperation with DreamWorks Animation, Universal Parks and Resorts and Comcast Labs.
The theater, two years in the making, was designed by Foster + Partners, a London-based architectural firm led by architect Norman Foster. It is ADA-compliant with wheelchair access and closed captioning devices. The development team rented the Fels Planetarium at The Franklin Institute Science Museum as part of their show development.
Comcast CEO Brian Roberts explained “The first 50 years for our company were about finance and entrepreneurship. If we’re going to thrive for the next 50 years, it’s going to be about innovation, media, and technology” (Stephens, 2019).
Immersion domes have been called the “next big thing” for live immersive performances [Csathy 2018]. Two companies have announced major brick-and-mortar developments that promise to kick off a rapidly expanding ecosystem of immersion dome theaters.
Vortex Immersion Media has announced a new project in development called the Vortex DomePlex (Roettgers, 2019) (Figure 20). This immersive arts and entertainment complex includes a 2500 seat performing arts CineTheater ™, two 1,000-capacity Vortex Dome events domes and smaller dome spaces for retail, experiential lounges and VR arcades. The facility is designed as a major exhibition venue for stand-up, sit-down and walk-through immersive experiences.
Vortex Immersion Media 宣布了一个名为 Vortex DomePlex 的创新项目（Roettgers, 2019）（见图 20）。这个沉浸式艺术和娱乐的综合设施将包括一个能容纳 2500 人的 CineTheater ™表演艺术剧院，两个能容纳 1000 人的 Vortex Dome 活动球幕，以及多个小型球幕空间，用于零售、体验休息室和 VR 游戏。该设施的目标是成为一个集站立式、坐式和步行穿越的沉浸式体验于一体的主要展览场所。
The CineTheater ™ features an advanced projection-mapped elevator stage, video-tracked theatrical lighting and 3D sound including infrasound and vibroacoustic chairs. The theater will be fitted with a multicamera 360 video capture and streaming system for domecasting and VR multicasting. An MMO metaverse version of the venue will encourage remote preand post-event engagement and community building around a wide range of entertainment themes.
CineTheater ™将配备先进的投影映射电梯舞台、视频跟踪剧场照明，以及包括次声波和振动声学椅子在内的 3D 音效系统。剧院还将安装多摄像头 360 视频捕捉和流媒体系统，用于球幕广播和 VR 多播。此外，一个 MMO 元宇宙版本的场馆将促进远程的预活动和后活动参与，以及围绕广泛的娱乐主题建立社区。
One of the Vortex Domes will serve as an after-party venue allowing story extension. It will be equipped with photonic go-go booths, elevator stages for DJ’s (dome jockey), full audience tracking and multi-VJ performance consoles. The open architecture design will allow artists to perform from their own servers which can be installed on site. Another Vortex Dome will serve as flex space to expand storytelling worlds into walk-through immersive experiences, host travelling exhibits and more.
Vortex DomePlex 的其中一个 Vortex Dome 将作为一个派对场所，允许故事在其中延伸。它将配备光子摇摆舞厅、DJ 的电梯舞台（球幕骑师）、全场跟踪和多 VJ 表演控制台。开放式的架构设计将使艺术家能够通过现场安装的自己的服务器进行表演。另一个 Vortex Dome 将作为一个灵活的空间，将故事世界扩展到步行穿越的沉浸式体验中，用于举办巡回展览等活动。
Vortex’s business model is designed to be flexible, accommodating an ever-evolving technical, artistic and consumer market. The company seeks to present family programming in the daytime, “digital cirque” entertainment residencies in the evening and interactive electronic entertainment for late night.
Madison Square Garden Entertainment has announced the MSG SPHERE, a 17,500 seat LED-based digital dome to be constructed in Las Vegas and London. The project – currently estimated at $1.7b, features a 152m LED-mapped spherical exterior and an interior LED dome screen. The project is currently under construction in Las Vegas.
麦迪逊广场花园娱乐公司宣布了 MSG SPHERE，这是一个 17,500 座位的基于 LED 的数字球幕，将在拉斯维加斯和伦敦建造。该项目目前估计耗资 17 亿美元，特色是一个 152 米 LED 映射的球形外观和一个室内 LED 球幕屏幕。该项目目前正在拉斯维加斯建设中。
未来研究 FUTURE RESEARCH
Immersion domes suggest a wide range of academic and commercial research opportunities spanning neuroscience and cognitive science, computer graphics and interactive techniques.
Cognitive research is needed to better understand the effect of immersive media on consciousness and it’s use in storytelling and delivering impact. It is possible that unique brain states such as the pilomotor reflex and the release of pleasurable brain chemicals such a dopamine, serotonin, oxytocin and endorphins could be optimized in highly immersive and interactive experiences (Salimpoor, Benovoy, Larcher, Dagher & Zatorre, 2011; Manninen, et al. 2017). And how can audience biometrics combined with AI be used to optimize these responses?
The economics of dome theaters allows the installation of GPU supercomputers for real-time raytracing and deep learning algorithms. What new experiences can be offered with such computing power? If we can specifically identify audience members and—through their opting in—can access their “big data” social media assets, how can AI use these assets to enhance the audience experience? And how can AI be used to assist the interpretation of music through immersive visuals, or vice versa?
Rendering, animation and computing algorithms are often biased towards planar imagery and cartesian coordinates. What shortcuts might be possible when serving graphics exclusively for spherical environments?
The dome is effectively a portal into cyberspace. XR technologies can bring cyberspace into the real space of the theater as well. Stage performances within immersive environments are greatly enhanced by tracking both props and performers and altering the immersive environment in response to their movements. Lighting and textures can be mapped onto moving performers, providing avatar-like capabilities to transform. Performers can be volumetrically scanned and placed into MMO metaverses and role-playing games.
And what AR/VR/XR/360 interfaces and devices and applications can be developed for in-dome or at-home audience participation? Can a billion people attend a single immersion dome event?
Arts and entertainment in digital immersion domes have been steadily growing, first appearing in domes intended for astronomy and space science and giant screen documentary films. Market growth is evidenced by the success of Mesmerica, the Vortex Dome, SAT, IAIA’s Digital Dome and the more recent success of the Wisdome Art Park plus the success of Pharos which received praise from Billboard and Childish Gambino’s many fans. Recent announcements of MSG Sphere and the Vortex DomePlex indicate that arts and entertainment domes are on a growth trajectory.
If digital arts and entertainment dome networks are to be established there will need to be standards in dome designs. Two general venue designs have been suggested for standardization efforts; the Multipurpose Dome and the CineTheater ™. A number of relevant design factors were reviewed to support such an effort. It is hoped that industry-wide cooperation will lead to a robust distribution network, allowing artists and producers to benefit from digital distribution of powerful immersive experiences.
It is also hoped that the power of this next-generation immersive format is used mindfully for positive societal impact. Digital domes are part of an immersive media ecosystem that will have profound and far-reaching impact on humanity.
Csathy, P. (2018). Fearless Media 2.0: An insider’s guide & call to action for today’s media 2.0 world & where it’s going. CREATV Media, Ch., 3(Part III), 269–271.
Gaston, L., Dougall, P., & Thompson, E. D. (2008). Methods for Sharing Stereo and Multichannel Recordings among Planetariums. Proceedings of the AES 124th Convention.
GOTO, Inc. (2016). Second full-sphere projection system installed at Fukushima Prefectural Government Environmental Creation Center. Retrieved from http://www.goto.co.jp/english/news/20160908/
Habe, H., Saeki, N., & Matsuyama, T. (2007). Inter-reflection compensation for immersive projection display. Proceedings of the IEEE International Workshop on Projector-Camera Systems (ProCams).
Kleiman, J. (2019). Evans & Sutherland launches DomeX LED display for fulldome planetariums and giant screen theaters. InPark Magazine. Retrieved from http://www.inparkmagazine.com/about/
Kuchera-Morin, J., Wright, M., Wakefield, G., Roberts, C., Adderton, D., Sajadi, B., … Majumder,
A. (2014). Immersive full-surround multi-user system design. Computers & Graphics, 40, 10–21. doi:10.1016/j.cag.2013.12.004
Lantz, E. (1997). Future Directions in Visual Display Systems. Guest Editor. Computer Graphics, 31(2), 38–45. doi:10.1145/271283.271301
Lantz, E. (2004). Fulldome Display Specifications: A Proposal. IPS 2004 Fulldome Standards Summit, Valencia, Spain.
Lantz, E. (2009, June). The Planetarium: A Transitional Animal. Planetarian, 38(2), 6–12.
Lantz, E. (2011, July). Planetarium of the Future. Curator, 54(3), 293–312. doi:10.1111/j.21516952.2011.00093.x
Lantz, E. (2018). From space to the stars: Ten years of arts and entertainment at The Vortex Dome-Los Angeles. Planetarian, 47(2), 22–28.
Manninen, S., Tuominen, L, Dunbar, R., Karjalainen, T., Hirvonen, J., Arponen, E., … Nummenmaa, L. (2017). Social Laughter Triggers Endogenous Opioid Release in Humans. The Journal of Neuroscience.
Zhang, Y., Shen, Y., Zhang, W., Zhu, Z., & Ma, P. (2019). Interactive spatial augmented reality system for Chinese opera. In Proceedings of SIGGRAPH 2019. Article 14. pp. 1-2. Retrieved from https://dl.acm.org/citation.cfm?id=3338566
当前情况下，后三点难以立即得出结论。我们尚不清楚 AI 视频将如何改变消费者行为，以及消费者如何重新定义质量；我们还不知道老牌企业会作何反应（尽管 我曾写过，我认为他们准备不足，这在很大程度上是由于其运作的复杂生态系统，包括人才、经纪公司、工会等）；我们也不清楚传统专业视频制作的“高端”市场最终能有多大规模或多持久。
再次引用上述对比：在视频分发领域的颠覆过程中，需要挖掘大量土地、铺设光纤和建造发射塔；而 AI 视频技术的进步则无需任何这类实体设施的建设。
尽管理论上可能发展很快，但并不代表颠覆一定会如此发生。在 Sora 出现之前，人们认为 AI 视频可能会遇到某种技术瓶颈，这也是可以想象的。并非所有的 AI 问题都必然有解。由于我们尚未真正理解人类智能的基础，所以也许我们永远无法实现通用人工智能（AGI）。或许我们永远无法拥有真正的“5 级”自动驾驶，因为我们不愿接受（不可避免地存在的）非零风险，即自动驾驶车辆有时会致人死亡。然而，Sora 的成功表明，视频这一问题至少是能够找到解决方案的。
开源开发。 AI 在某种程度上是独特的，因为它的发展很大程度上是在公共领域进行的，即使是由私营公司推动的进步也是如此。（例如，支撑 OpenAI 的核心创新，源于谷歌率先提出的 Transformer 架构。）这可能是因为许多 AI 研究的根源在学术界，人们相信开源有利于公益并加速发展，或者由于处于前沿所带来的公关优势。无论原因为何，大量研究成果公开发布的趋势显然正在加快创新的步伐。
组合性。 基于上述观点，尽管 Sora 的技术论文并未完全揭示底层架构和模型的秘密所在，但其众多引用清楚表明它建立在许多先前进展的基础之上。这些进展包括但不限于 Transformer Diffusion 模型的概念、时空补丁技术、Diffusion 模型的改进，以及文本条件化（text conditioning）的进步。在软件领域，能够像乐高积木一样组合使用多种不同工具和模型的能力被称为“组合性（composability）”。许多 AI 模型设计成模块化形式，以便用户轻松地与其他工具结合或使用。这也同样加快了创新的速度。
规模效应。 2020 年，OpenAI 的研究人员发表了一篇名为 《神经语言模型的规模定律》（Scaling Laws for Neural Language Models） 的论文，指出模型规模（scale）的重要性超过了模型架构本身。研究发现，LLM（大规模语言模型）的性能会随着训练数据集大小、参数数量（模型复杂度的衡量指标）以及计算能力的增加而平滑提升。如图 12 所示，每个变量的规模与其模型误差之间存在明显的无界逆幂律关系（unbounded inverse power law relationship）。换句话说，规模效益并无明显的上限。这一点在随附论文的视频中生动展现出来。图 13 展示了在不同计算级别下，一只戴着蓝色帽子在雪地里玩耍的狗的三个视频片段。
]]>
]]>
]]>
]]>
