Back to Search Document details
12th Meeting: Macao, October 2018 2018-10-06 11:08
CE15: Summary report on palette mode
Authors: Yu-Chen Sun Yung-Hsuan Chao Xiaozhong Xu
Abstract
A summary of Core Experiment 15 (CE15) on palette mode is reported. Two tests are evaluated based on the common test conditions defined in JVET-K1035. All tests and crosscheck results are integrated in this report.
Meeting Notes
JVET-L0035 CE15: Summary report on palette mode [Y.-C. Sun, Y.-H. Chao, X. Xu]

The following tests in the table below were performed in CE15.

Test

Tester

Document

Tool description

Cross checker

15.1

Yu-Chen Sun (Alibaba)
Yung-Hsuan Chao (Qualcomm)

JVET-K0411

Palette mode in JVET-K0411

J. Ye (Tencent)

15.2

Yung-Hsuan Chao (Qualcomm)
Yu-Chen Sun (Alibaba)

HEVC-SCC

Palette mode in HEVC-SCC

Jianqing Zhu (Fujitsu)

CE15.1 and CE15.2 share the same software code base and can be switched by macro setting. The difference between two tests are summarized in the following table.

CE15.1

CE15.2

Index map coding

  1. Raster scan
  2. Escape mode is coded in INDEX mode
  3. Palette Index is signaled (TBC)
  4. PLT index(bypass), run type(CABAC), run length(CABAC), and escape sample (bypass) are coded in interleaved manner
  5. Escape sample is coded using TBC (QP dependent)
  6. Escape sample: encode Y/ Cb / Cr together for each escape sample
  1. Horizontal / Vertical traverse scan (flag)
  2. Escape mode is coded in INDEX/COPY_ABOVE mode
  3. Redundancy removal before signalling index (TBC) :
    if previous pixel is INDEX: curIDX prevIDX
    if previous pixel is COPY_ABOVE: curIDX aboveIDX
  4. Encode PLT index (bypass) for whole CU
    -> run type (CABAC) and run length (CABAC) interleaved
    -> escape samples for whole CU (bypass)
  5. Escape sample is coded with Exponential-Golomb coding
  6. Encode Y for whole CU -> Cb for whole CU -> Cr for whole CU.

Syntax inference

  1. Signal palette info regardless of palette table size
  1. If palette table size <= 1:
    PLT index = 0
    run type = INDEX
    scanning: horizontal
    run length = number of pixels in the CU
    if palette table size > 1:
    signalling palette info

Bug fix

  1. Bug fix in escape sample coding: fix the issue in division (only affect low QP)

The results of the CE15.1 and CE15.2 software released Sep. 17 are denoted as “15.1” and “15.2”. Later, the proponents provided an encoding-only update, and the results are denoted as “15.1*” and “15.2*. The software with encoding only change is updated roughly 12 hours after the deadline (PST). The anchor is BMS-2.0.1 with VTM cfg.

All Intra Main 10

VTM_config

Test#

Y

U

V

EncT

DecT

CTC overall

15.1

0.11%

0.12%

0.11%

111%

104%

15.2

0.09%

0.12%

0.11%

111%

102%

15.1*

0.11%

0.12%

0.11%

104%

99%

15.2*

0.09%

0.13%

0.12%

108%

104%

Class F

15.1

-10.27%

-7.97%

-7.95%

122%

92%

15.2

-11.51%

-9.19%

-9.23%

122%

96%

15.1*

-10.27%

-7.97%

-7.99%

116%

91%

15.2*

-11.52%

-9.23%

-9.20%

119%

96%

SCC 1080p

15.1

-30.44%

-25.20%

-25.23%

138%

70%

15.2

-33.48%

-28.20%

-28.24%

130%

68%

15.1*

-30.44%

-25.18%

-25.22%

132%

68%

15.2*

-33.48%

-28.20%

-28.25%

129%

67%

Random Access Main 10

VTM_config

Test#

Y

U

V

EncT

DecT

CTC overall

15.1

0.12%

0.08%

0.11%

110%

102%

15.2

0.11%

0.10%

0.16%

113%

102%

15.1*

0.13%

0.16%

0.16%

106%

100%

15.2*

0.17%

0.26%

0.23%

107%

103%

Class F

15.1

-7.95%

-7.33%

-7.76%

117%

99%

15.2

-8.89%

-8.33%

-8.76%

114%

97%

15.1*

-7.80%

-7.21%

-7.63%

107%

95%

15.2*

-8.72%

-8.07%

-8.63%

108%

98%

SCC 1080p

15.1

-14.56%

-13.46%

-13.17%

124%

92%

15.2

-17.06%

-16.01%

-15.69%

122%

88%

15.1*

-13.93%

-12.76%

-12.45%

100%

88%

15.2*

-16.44%

-15.30%

-14.99%

101%

88%

Low Delay B Main 10

VTM_config

Test#

Y

U

V

EncT

DecT

CTC overall

15.1

0.17%

0.06%

-0.04%

112%

103%

15.2

0.11%

0.20%

-0.11%

112%

103%

15.1*

0.15%

0.10%

-0.12%

105%

101%

15.2*

0.13%

0.18%

-0.07%

108%

103%

Class F

15.1

-4.77%

-4.08%

-4.89%

117%

105%

15.2

-5.67%

-5.07%

-6.25%

112%

100%

15.1*

-4.46%

-3.18%

-4.18%

107%

100%

15.2*

-5.22%

-4.71%

-5.88%

105%

100%

SCC 1080p

15.1

-6.42%

-6.17%

-5.56%

123%

100%

15.2

-8.94%

-8.89%

-8.21%

121%

96%

15.1*

-6.00%

-5.79%

-5.01%

98%

94%

15.2*

-8.64%

-8.59%

-7.77%

100%

94%

For CTC (camera-content video), some small loss occurs. This is somehow expected as it is known that palette mode is not matching the properties of natural video.

It is pointed out that 15.2 is different from HEVC, as it uses separate palette for luma and chroma in intra slices (when dual tree is enabled), whereas HEVC-SCC uses a combined palette. This is however necessary, as dual tree enforces different CU shapes.

Additional results were shown in a powerpoint presentation (v4 of JVET-L0035) that additonal gain (approx. 1% for RA) is possible when enabling separate trees also for inter slices. It is requested to submit those results as a CE related contribution.

Why is it necessary to simplify palette compared to HEVC? At CU level, HEVC palette mode is not the most complex element. However, in hardware it requires additional building blocks separate from the other elements. In mobile applications, the saving of computing power when palette is invoked may be interesting as well.

At the current status, it would be premature to adopt a tool that has specific advantage only for screen content to VVC. 15.2 might be a good candidate for BMS, if it would still exist. Otherwise, it shall be used as reference in the upcoming CE cycle. The continuing CE shall provide results both with and without CPR enabled (same reference version as used in CE8). Several experts had the opinion that the version with separate trees for inter should be a better reference. This should be further discussed along with the aforementioned CE related contribution.

Aspect for plenary discussion: How much effort should we currently put into tools that mainly have advantage for screen content?

Additional results combined with CPR (from BMS)

All Intra Main 10

VTM_config

Test#

Y

U

V

EncT

DecT

Class F

8.3.1a (curCTU)

-11.61%

-11.82%

-11.87%

155%

102%

15.1

-10.27%

-7.97%

-7.95%

122%

92%

15.2

-11.51%

-9.19%

-9.23%

122%

96%

15.1 + 8.3.1a (curCTU)

-14.51%

-13.10%

-13.21%

169%

94%

15.2 + 8.3.1a (curCTU)

-15.33%

-14.05%

-14.14%

168%

94%

SCC 1080p

8.3.1a (curCTU)

-36.51%

-35.89%

-36.20%

148%

99%

15.1

-30.44%

-25.20%

-25.23%

138%

70%

15.2

-33.48%

-28.20%

-28.24%

130%

68%

15.1 + 8.3.1a (curCTU)

-43.27%

-39.84%

-40.03%

157%

75%

15.2 + 8.3.1a (curCTU)

-44.65%

-41.13%

-41.32%

155%

73%

Random Access Main 10

VTM_config

Test#

Y

U

V

EncT

DecT

Class F

8.3.1a (curCTU)

-9.63%

-9.93%

-9.88%

115%

102%

15.1

-7.95%

-7.33%

-7.76%

117%

99%

15.2

-8.89%

-8.33%

-8.76%

114%

97%

15.1 + 8.3.1a (curCTU)

-11.36%

-11.21%

-11.23%

126%

101%

15.2 + 8.3.1a (curCTU)

-12.04%

-11.86%

-12.15%

128%

102%

SCC 1080p

8.3.1a (curCTU)

-22.76%

-22.82%

-23.10%

110%

97%

15.1

-14.56%

-13.46%

-13.17%

124%

92%

15.2

-17.06%

-16.01%

-15.69%

122%

88%

15.1 + 8.3.1a (curCTU)

-25.54%

-24.91%

-24.82%

123%

94%

15.2 + 8.3.1a (curCTU)

-26.86%

-26.21%

-26.10%

124%

93%

Palette mode gives still additional gain when combined with CPR. Its standalone gain is smaller than the standalone gain of CPR.

References:
JCTVC-K1035 JCTVC-W1005 JVET-K0411 JVET-K1035
ARCHIVE v4 :
._CE15_report.pptx ._JVET-L0035.docx ._JVET-L0035_CE15_results_summary.xlsx ._JVET-L0035_v2.docx ._JVET-L0035_v3.docx ._JVET-L0035_v4.docx ._JVET-L0035_v4_clean.docx CE15_report.pptx JVET-L0035.docx JVET-L0035_CE15_results_summary.xlsx JVET-L0035_v2.docx JVET-L0035_v3.docx JVET-L0035_v4.docx JVET-L0035_v4_clean.docx
Decisions
Palette mode gives still additional gain when combined with CPR. Its standalone gain is smaller than the standalone gain of CPR.
Citation