JVET-P1033 Report of BoG on CE5-related loop filter proposals [A. Norkin, C.-Y. Chen]

Presented Thursday 10 Oct. 1415 in Track B

The BoG reviewed proposals related to the loop filter topics (CE5-related). This report includes the recommendations of the BoG for adopting or further studying of proposals.

The BoG met during the following times:

• Tuesday Oct. 8: 9:00-13:00, 14:30-21:00
• Wednesday Oct. 9: 11:00-13:30, 15:00-20:10
• Thursday Oct. 10: 9:00-12:30

Recommendations of BoG:

Deblocking filter

It is recommended to adopt proposal JVET-P0043 (the specification text). The software attached to the proposal needs to be fixed.

Decision: Adopt JVET-P0043. This is correcting some inconsistency of deblocking in case of affine 4x4 and TPM block edges, when they coincide with TU boundary. Text was confirmed to be appropriate, aligned software still needs to be uploaded.

It is suggested to study proposal 1 of JVET-P0161 (and similar proposals, see under notes of JVET-P0161) in the CE. This would be either or two test cases (depending if proponents of JVET-P0586 decided to join the other test or have a separate test).

In track A, it is discussed that this would require storing information about presence of weighted prediction and imposing them in the decision of deblocking.

Also test JVET-P0269 method 1 in CE.

Test in the CE of JVET-P0441 (where subjective improvements are expected). This would change some conditions to not apply deblocking at affine inner boundaries unless these are at TU boundaries. This requires one more condition check, so it is not a simplification.

Test JVET-P0411 in CE (additional boundary condition that avoids oversmoothing of long DBF). Not a simplification.

It is strongly recommended that, if several proposals propose the same idea, only one of them should be tested. It is further strongly recommended that, if a given proposal suggests multiple solutions to one problem, only one of these solutions shall be tested, as proponent should know best which of them is preferable. It shall be decided already during the setup of the CE, which methods and how many cases would be up to subjective viewing, as otherwise this is not manageable. Furthermore, crosscheckers are mandated to perform subjective viewing prior to the meeting and report if they were able to recognize differences.

It is noted that proposals that would make deblocking somewhat more complicated require a proof to have an improvement in visual quality. It is further noted that in the interest of stability of the DBF design (in particular also the stability of the specification text, it should be demonstrated that a proposal to be adopted should really be solving a problem.

Adopt method 1 of JVET-P0571

Decision (BF/SW): Adopt JVET-P0571 method 1 – align SW with spec text.

Adopt solution 2 of JVET-P1001: Current deblocking just uses luma QP for chroma deblocking decision. It is proposed to modify this by considering chroma QP offsets. It is agreed that this is a reasonable change, and complexity-wise (additional line buffer) should be OK.

Decision: Adopt JVET-P1001 solution 2

ALF

Recommendations:

• Adopt method 1 of JVET-P0158, applying ALF VB to all CTU in a picture, including the bottom CTU row.
• Adopt method 2 in JVET-P0551, which is also method 4 in JVET-P0157 and first aspect of JVET-P0053. The software is from JVET-P0551.
• Use the check for the unavailable samples as in JVET-P0552 (and same as in JVET-P0492). This was further discussed in a BoG Thursday evening.

Decision: Adopt JVET-P1038 (combined text and software from the modifications above); agreed in closing plenary (GJS & JRO) at 1100.

Recommended to adopt JVET-P0162: Remove alf_ctb_first_use… flag

Decision (cleanup): Adopt JVET-P0162

Recommended to adopt JVET-P0164 (cleanup of APS syntax)

Decision (cleanup): Adopt JVET-P0164 method 2.

BoG recommendation: Adopt method 2 of JVET-P0554 (align text with software).

Decision (BF/text): Adopt JVET-P0554 method 2.

BoG recommendation: The BoG recommended fixing the specification text to reflect the desired behaviour (using the correct samples for padding), as implemented in software.

Decision (BF/text): Adopt JVET-P0665 (preferable text from zip version 2, but left to the discretion of the editors if this is the best way for fixing this problem).

In JVET-P0505, it is pointed out that there is a mismatch between text and software in regard to clipping values for chroma: Whereas the VVC6 text derives the clipping values using the same table for luma and chroma, the software still has two different tables where for chroma this would not match the table from the text (this is due to the fact that it was never implemented, in the meantime ticket 551 was issued). The contribution found that if this was fixed, losses would occur in chroma for the 8 bit internal bitdepth case.

The proponents suggest to define another table that would be used for luma and chroma, which would just replace the current table, have very similar performance for bit depth >8 (up to 12 was tested), which would heal the problem with internal 8 bitdepth

BoG recommendation: Adopt the v5 of JVET-P0505.

Decision: Adopt JVET-P0505.

CCALF

BoG recommendation: Test JVET-P0106 in the CE. Since the shape is highly asymmetric, a test on other chroma locations is required.

BoG recommendation: Study JVET-P0173 in the CE.

BoG recommendation: Test C6-2 of JVET-P0251 in the CE.

BoG recommendation: test JVET-P0372 in the CE.

BoG recommendation: test method 3 of JVET-P0557 in the CE.

BoG recommendation: test method 5 of JVET-P0558 the CE.

BoG recommendation: Test the results of JVET-P0666 with limiting the coefficient range to [-8, 8]. The proponents are advised to discuss offline what coefficient bit depth (range) to test in the CE.

Agreed to test these in CE.

JVET-P1008

Changes to CE5-2:

Proposed simplifications:

• Complexity reduction
  • Reduce the number of multipliers in the filter operation
  • Limit dynamic range of coefficient to 6-bits
  • Allow for sharing of multipliers with chroma ALF
• Alignment with ALF
  • Limit filter selection to signalling at a CTU level
  • Removal of the temporal layer coefficient buffers
  • Use of symmetric line selection at virtual boundary

Additionally, as the simplifications reduce coding efficiency, the authors also propose to use up to 4 filters per chroma component (called option 2 below)

• Disallow the use of both CC-ALF and chroma ALF in a CTB
• For each chroma CTB
  • Signal CC-ALF on/off flag
  • If CC-ALF enabled, chroma ALF is disabled
  • Otherwise, signal chroma ALF on/off flag

Benefit: Number of multiplications per pixel are reduced from 19 (called option 1, if CCALF is operated on top of current chroma ALF) to 16 (called option 2, if only one of them is used in a certain region). Number is 15 in current ALF.

The method tested in CE5-2 had 22 multiplications.

Results for Option 1 on RA are Y=0.17%, Cb = -11.53%, Cr -10.99%. This is slightly better than the CE5-2.1 result, same for AI, however loss (relative to CE) occurs for LB

Results for Option 2 were incomplete.

It is suggested to investigate if there might be more severe subjective quality impact towards higher QP values than those used in the CE5-2 subjective test at this meeting. This is definitely something that would be needed to be investigated, to be sure that the VTM encoder in its default settings would not cause problems when used for upcoming VVC verification tests. This should be performed as part of the CE.

It is generally agreed that Option 1 would be implementable in hardware. It is not obvious that option 2 would have advantage in terms of implementation, as it might not be possible to share same circuitry (may be depending on architecture).

An informal viewing session should be organized showing CTC sequences at QP37, to demonstrate that there is no problem with visual quality also for the modified method (JVET-P1008 option 1).

It was planned to adopt the proposal (option 1) if nobody who had been in the viewing session raised an objection in terms of visual artefacts.

If not adopted to VVC7, this should be used as anchor in the CE.

There was no action suggested for other categories of proposals.

CCALF was further discussed in the closing plenary 1105 (GJS & JRO).

Some informal viewing had been conducted; participants saw some differences but did not seem to have a clear preference. Further viewing with high QP was suggested.

Some simplifications were suggested to be tested.

Testing with HDR was suggested.

It was agreed to have this as an anchor in a CE.

JVET-P1033 Report of BoG on CE5-related loop filter proposals [A. Norkin, C.-Y. Chen]

See section 6.5.

List of actions taken affecting the draft text of VVC, the VTM, and 360Lib

The following is a summary, in the form of a brief list, of the actions taken at the meeting that affect the text of the VVC draft text, VTM or 360Lib description. Both technical and editorial issues are included. This list is provided only as a summary – details of specific actions are noted elsewhere in this report and the list provided here may not be complete and correct. The listing of a document number only indicates that the document is related, not that it was adopted in whole or in part. The description given in the “Tool” column is a best effort for the sake of understanding but may not precisely reflect the functionality of the tool. It is also noted that in cases where several contributions proposed the same method, usually only one of the is listed as adoption below; refer to the meeting notes about the adoption to see which other contributions are related.

JVET-P0983 impact may need clarification in list. Benefit would be low, only saves runtime for small resolutions.

Project planning

Core experiment planning

From Track A, initial planning Thursday:

• Discontinue CE3, CE6, CE7
• Continue CE8 – Could be renamed as “Palette mode”
• Have new CE on lossless (T. Nguyen et al.)
• Continue CE5 – both deblocking and CCALF aspects.

See final planning under JVET-P2021…P2025.

Drafting of specification text, encoder algorithm descriptions, and software

The following agreement has been established: the editorial team has the discretion to not integrate recorded adoptions for which the available text is grossly inadequate (and cannot be fixed with a reasonable degree of effort), if such a situation hypothetically arises. In such an event, the text would record the intent expressed by the committee without including a full integration of the available inadequate text.

Plans for improved efficiency and contribution consideration

The group considered it important to have the full design of proposals documented to enable proper study.

Adoptions need to be based on properly drafted working draft text (on normative elements) and HM encoder algorithm descriptions – relative to the existing drafts. Proposal contributions should also provide a software implementation (or at least such software should be made available for study and testing by other participants at the meeting, and software must be made available to cross-checkers in EEs).

Suggestions for future meetings included the following generally-supported principles:

• No review of normative contributions without draft specification text
• VTM algorithm description text is strongly encouraged for non-normative contributions
• Early upload deadline to enable substantial study prior to the meeting
• Using a clock timer to ensure efficient proposal presentations (5 min) and discussions

The document upload deadline for the next meeting was planned to be Tuesday 31 December 2019.

As general guidance, it was suggested to avoid usage of company names in document titles, software modules etc., and not to describe a technology by using a company name.

General issues for experiments

Group coordinated experiments have been planned as follows:

• “Core experiments” (CEs) are the coordinated experiments on coding tools which are deemed to be interesting but require more investigation and could potentially become part of the draft standard by the next meeting.
• A CE is a test of a specific fully described technology in a specific agreed way. It is not a forum for thinking of new ideas (like an AHG). The CE coordinators are responsible for making sure tha the CE description is complete and correct and has adequate detail. Reflector discussions about CE description clarity and other aspects of CE plans are encouraged.
• A description of each experiment is to be approved at the meeting at which the experiment plan is established. This should include the issues that were raised by other experts when the tool was presented, e.g., interference with other tools, contribution of different elements that are part of a package, etc. The experiment description document should provide the names of individual people, not just company names.
• Software for tools investigated in a CE will be provided in one or more separate branches of the software repository. Each CE will have a “fork” of the software, and within the CE there may be multiple branches established by the CE coordinator. The software coordinator will help coordinate the creation of these forks and branches and their naming. All JVET members have read access to the CE software branches (using shared read-only credentials; see the AHG3 report for further detail).
• During the experiment, revisions of the experiment plans can be made, but not substantial changes to the proposed technology.
• The CE description must match the CE testing that is done. The CE description needs to be revised if there has been some change of plans.
• The CE summary report must describe any changes that were made in the process of finalizing the CE.
• By the next meeting it is expected that at least one independent cross-checker will report a detailed analysis of each proposed feature that has been tested and confirm that the implementation is correct. Commentary on the potential benefits and disadvantages of the proposed technology in cross-checking reports is highly encouraged. Having multiple cross-checking reports is also highly encouraged (especially if the cross-checking involves more than confirmation of correct test results). The reports of cross-checking activities may (and generally should) be integrated into the CE report rather than submitted as separate documents.

It is possible to define sub-experiments within particular CEs, for example designated as CEX.a, CEX.b, etc., where X is the basic CE number.

As a general rule, it was agreed that each CE should be run under the same testing conditions using one software codebase, which should be based on the group test model software codebase. An experiment is not to be established as a CE unless there is access given to the participants in (any part of) the CE to the software used to perform the experiments.

The general agreed common conditions for single-layer coding efficiency experiments are described in the output document JVET-N1010.

Experiment descriptions should be written in a way such that it is understood as a JVET output document (written from an objective “third party perspective”, not a proponent perspective – e.g. not referring to methods as “improved”, “optimized”, etc.). The experiment descriptions should generally not express opinions or suggest conclusions – rather, they should just describe what technology will be tested, how it will be tested, who will participate, etc. Responsibilities for contributions to CE work should identify individuals in addition to company names.

CE descriptions contain a basic description of the technology under test, but should not contain excessively verbose descriptions of a technology (at least not unless the technology is not adequately documented elsewhere). Instead, the CE descriptions should refer to the relevant proposal contributions for any necessary further detail. However, the complete detail of what technology will be tested must be available – either in the CE description itself or in documents that are referenced in the CE description that are also available in the JVET document archive.

Any technology must have at least one cross-check partner to establish a CE – a single proponent is not enough. It is highly desirable have more than just one proponent and one cross-checker.

CE software is available to read access by all accredited members, and CE software coordination is conducted, as described in the AHG3 report.

Some agreements relating to CE activities were established as follows:

• Only qualified JVET members can participate in a CE.
• Participation in a CE is possible without a commitment of submitting an input document to the next meeting. Participation is requested by contacting the CE coordinator.
• All software, results, and documents produced in the CE should be announced and made available to JVET in a timely manner.
• A JVET CE reflector will be established and announced on the main JVET reflector. Discussion of logistics arrangements, exchange of data, minor refinement of the test plans, and preparation of documents shall be conducted on the JVET CE reflector, with subject lines prefixed by “[CEx: ]”, where “x” is the number of the CE. All substantial communications about a CE other than such details shall take place on main JVET reflector. In the case that large amounts of data are to be distributed, it is recommended to send a link to the data rather than the data itself, or upload the data as an input contribution to the next meeting.

General timeline for CEs

T1= 3 weeks after the JVET meeting: To revise the CE description and refine questions to be answered. Questions should be discussed and agreed on JVET reflector. Any changes of planned tests after this time need to be announced and discussed on the JVET reflector. Initially assigned description numbers shall not be changed later. If a test is skipped, it is to marked as “withdrawn”.

T2 = Test model software release + 2 weeks: integration of all tools into a separate CE branch of the VTM is completed and announced to JVET reflector.

• Initial study by cross-checkers can begin.
• Proponents may continue to modify the software in this branch until T3
• 3rd parties are encouraged to study and make contributions to the next meeting with proposed changes

T3: 3 weeks before the next JVET meeting or T2 + 1 week, whichever is later: Any changes to the CE test branches of the software must be frozen, so the cross-checkers can know exactly what they are cross-checking. A software version tag should be created at this time. The name of the cross-checkers and list of specific tests for each tool under study in the CE plan description shall be documented in an updated CE description by this time.

T4: Regular document deadline – 1 week: CE contribution documents including specification text and complete test results shall be uploaded to the JVET document repository (particularly for proposals targeting to be promoted to the draft standard at the next meeting).

The CE summary reports shall be available by the regular deadline. This shall include documentation about crosscheck of software, matching of CE description and confirmation of the appropriateness of the text change, as well as sufficient crosscheck results to create evidence about correctness (crosscheckers must send this information to the CE coordinator at least 3 days ahead of the document deadline). Furthermore, any deviations from the timelines above shall be documented. The numbers used in the summary report shall not be changed relative to the description document.

CE reports may contain additional information about tests of straightforwared combinations of the identified technologies. Such supplemental testing needs to be clearly identified in the report if it was not part of the CE plan.

New branches may be created which combine two or more tools included in the CE document or the VTM (as applicable).

It is not necessary to formally name cross-checkers in the initial version of the CE description document. To adopt a proposed feature at the next meeting, we would like see comprehensive cross-checking done, with analysis that the description matches the software, and recommendation of value of the tool given tradeoffs.

The establishment of a CE does not indicate that a proposed technology is mature for adoption or that the testing conducted in the CE is fully adequate for assessing the merits of the technology, and a favourable outcome of CE does not indicate a need for adoption of the technology.

Availability of spec text is important to have a detailed understanding of the technology and also to judge what its impact on the complexity of the spec will be. There must also be sufficient time to study it in detail. CE contributions without sufficiently mature draft spec text in the CE input document should not be considered for adoption.

Lists of participants in CE documents should be pruned to include only the active participants. Read access to software will be available to all members.

Software development and anchor generation

The planned timeline for software releases was established as follows:

• VTM7.0 will be released by 2019-11-11 including all adoptions necessary for CTC and CE basis references. VTM7.1 with non-CTC adoptions will be released later. Further versions of VTM may be released for additional bug fixing, as appropriate.
• Preparation of the VTM software will include immediate removal of macros that were added in the previous meeting cycle. The software coordinator has the discretion to retain some such macros.
• No change of of 360lib or HDRTools was noted in response to actions taken at this meeting.

Establishment of ad hoc groups

The ad hoc groups established to progress work on particular subject areas until the next meeting are described in the table below. The discussion list for all of these ad hoc groups was agreed to be the main JVET reflector (jvet@lists.rwth-aachen.de).

Output documents

The following documents were agreed to be produced or endorsed as outputs of the meeting. Names recorded below indicate the editors responsible for the document production. Where applicable, dates of planned finalization and corresponding parent-body document numbers are also noted.

It was reminded that in cases where the JVET document is also made available as MPEG output document, a separate version under the MPEG document header should be generated. This version should be sent to GJS and JRO for upload.