azalea/kotlin-fork
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Unmute non-flaky tests and mark some flaky tests with FLAKY label

Browse Source 
KTI-216
This commit is contained in:
Yunir Salimzyanov
2020-05-20 10:20:01 +03:00
parent 4e732b7ba2
commit ea71a57fac
6 changed files with 40 additions and 115 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/mute-common.csv
+8 -13
View File
@@ -13,26 +13,26 @@ org.jetbrains.kotlin.checkers.JavaAgainstKotlinSourceCheckerWithoutUltraLightTes
"org.jetbrains.kotlin.gradle.NewMultiplatformProjectImportingTest.testDetectAndroidSources[1: Gradle-4.9, KotlinGradlePlugin-latest stable]", Gradle Import Tests,,
"org.jetbrains.kotlin.gradle.NewMultiplatformProjectImportingTest.testDetectAndroidSources[3: Gradle-5.6.4, KotlinGradlePlugin-latest stable]", Gradle Import Tests,,
"org.jetbrains.kotlin.gradle.NewMultiplatformProjectImportingTest.testDetectAndroidSources[3: Gradle-5.6.4, KotlinGradlePlugin-latest stable]", Gradle Import Tests,,
"org.jetbrains.kotlin.gradle.NewMultiplatformProjectImportingTest.testSingleAndroidTarget[1: Gradle-4.9, KotlinGradlePlugin-latest stable]", Stable on windows,,
"org.jetbrains.kotlin.gradle.NewMultiplatformProjectImportingTest.testSingleAndroidTarget[3: Gradle-5.6.4, KotlinGradlePlugin-latest stable]", Stable on windows,,
"org.jetbrains.kotlin.gradle.NewMultiplatformProjectImportingTest.testSingleAndroidTarget[1: Gradle-4.9, KotlinGradlePlugin-latest stable]", Stable on windows,, FLAKY
"org.jetbrains.kotlin.gradle.NewMultiplatformProjectImportingTest.testSingleAndroidTarget[3: Gradle-5.6.4, KotlinGradlePlugin-latest stable]", Stable on windows,, FLAKY
org.jetbrains.kotlin.idea.caches.resolve.MultiPlatformHighlightingTestGenerated.testJvmKotlinReferencesCommonKotlinThroughJavaDifferentJvmImpls, Always red,,
org.jetbrains.kotlin.idea.codeInsight.gradle.GradleBuildFileHighlightingTest.testKtsInJsProject[1: with Gradle-5.6.4], ERROR: Diagnostic's list should be empty,,
org.jetbrains.kotlin.idea.codeInsight.gradle.GradleBuildFileHighlightingTest.testKtsInJsProject[0: with Gradle-4.9], ERROR: Diagnostic's list should be empty,,
org.jetbrains.kotlin.idea.codeInsight.gradle.GradleBuildFileHighlightingTest.testKtsInJsProject, ERROR: Diagnostic's list should be empty,,
org.jetbrains.kotlin.idea.codeInsight.gradle.GradleConfiguratorTest.testAddLibraryGSK[1: with Gradle-5.6.4], unresolved kotlinModule,,
org.jetbrains.kotlin.idea.codeInsight.gradle.GradleConfiguratorTest.testAddNonKotlinLibraryGSK[1: with Gradle-5.6.4], unresolved kotlinModule,,
org.jetbrains.kotlin.idea.codeInsight.gradle.GradleConfiguratorTest.testAddTestLibraryGSK[1: with Gradle-5.6.4], unresolved kotlinModule,,
org.jetbrains.kotlin.idea.codeInsight.gradle.GradleConfiguratorTest.testConfigure10[1: with Gradle-5.6.4], absent test data,,
org.jetbrains.kotlin.idea.codeInsight.gradle.GradleConfiguratorTest.testConfigureGSK[1: with Gradle-5.6.4], absent test data,,
org.jetbrains.kotlin.idea.codeInsight.gradle.GradleConfiguratorTest.testConfigureGradleKtsKotlinDevVersion[1: with Gradle-5.6.4], absent test data,,
org.jetbrains.kotlin.idea.codeInsight.gradle.GradleConfiguratorTest.testConfigureKotlinDevVersion[1: with Gradle-5.6.4], absent test data,,
org.jetbrains.kotlin.idea.codeInsight.gradle.GradleConfiguratorTest.testConfigureKotlinWithPluginsBlock[1: with Gradle-5.6.4], absent test data,,
org.jetbrains.kotlin.idea.completion.test.JvmSmartCompletionTestGenerated.testFilterTo, KT-32919 Broken in 71cb65c064c7 or 2d5a05466dfa,,
org.jetbrains.kotlin.idea.configuration.ConfigureKotlinInTempDirTest.testKotlincExistsNoSettingsLatestRuntimeNoVersionAutoAdvance,,, FLAKY
org.jetbrains.kotlin.idea.configuration.ConfigureKotlinInTempDirTest.testNoKotlincExistsNoSettingsRuntime10,,, FLAKY
org.jetbrains.kotlin.idea.configuration.ConfigureKotlinTest.testJava9WithModuleInfo, ERROR: Require directive for kotlin.stdlib is expected,,
org.jetbrains.kotlin.idea.configuration.GradleConfigureProjectByChangingFileTestGenerated.Gradle.testWithJava9ModuleInfo, Probably test data should be updated,,
org.jetbrains.kotlin.idea.debugger.test.CoroutineDumpTestGenerated.testNoCoroutines, Stable on windows,,
org.jetbrains.kotlin.idea.debugger.test.CoroutineDumpTestGenerated.testThreeCoroutines, Stable on windows,,
org.jetbrains.kotlin.idea.debugger.test.CoroutineDumpTestGenerated.testTwoDumps, Stable on windows,,
org.jetbrains.kotlin.idea.debugger.test.CoroutineDumpTestGenerated.testNoCoroutines, Stable on windows,, FLAKY
org.jetbrains.kotlin.idea.debugger.test.CoroutineDumpTestGenerated.testThreeCoroutines, Stable on windows,, FLAKY
org.jetbrains.kotlin.idea.debugger.test.CoroutineDumpTestGenerated.testTwoDumps, Stable on windows,, FLAKY
org.jetbrains.kotlin.idea.decompiler.stubBuilder.BuiltInDecompilerConsistencyTest.testSameAsClsDecompilerForCompiledBuiltInClasses, No classes with this name found: kotlin.ArithmeticException,, FLAKY
org.jetbrains.kotlin.idea.decompiler.stubBuilder.BuiltInDecompilerTest.testBuiltInStubTreeEqualToStubTreeFromDecompiledText, No classes with this name found: kotlin.ArithmeticException,, FLAKY
org.jetbrains.kotlin.idea.highlighter.HighlightingTestGenerated.testSuspend, Always red,,
@@ -40,12 +40,11 @@ org.jetbrains.kotlin.idea.highlighter.NoErrorsInStdlibTest.testNoErrors, Always
org.jetbrains.kotlin.idea.maven.KotlinMavenInspectionTestGenerated.testWrongPhaseExecution, Always red,,
org.jetbrains.kotlin.idea.maven.MavenMigrateTest.testMigrateApiAndLanguageVersions, flaky on windows,, FLAKY
org.jetbrains.kotlin.idea.maven.configuration.MavenConfigureProjectByChangingFileTestGenerated.Jvm.testWithJava9ModuleInfo, Probably test data should be updated,,
org.jetbrains.kotlin.idea.quickfix.DeprecatedSymbolUsageFixSpecialTest.testDefaultParameterValuesFromLibrary, Always red because of additional import,,
org.jetbrains.kotlin.idea.quickfix.DeprecatedSymbolUsageFixSpecialTest.testDefaultParameterValuesFromLibrary, Always red because of additional import,, FLAKY
org.jetbrains.kotlin.idea.quickfix.QuickFixMultiModuleTestGenerated.CreateActual.testWithTestDummy, Very flaky on teamcity,, FLAKY
org.jetbrains.kotlin.idea.quickfix.QuickFixTestGenerated.CreateFromUsage.CreateClass.CallExpression.testCallInWhenEntry, KT-35728,,
org.jetbrains.kotlin.idea.quickfix.QuickFixTestGenerated.Libraries.testJunit, ERROR: Couldn't find the following libraries,,
org.jetbrains.kotlin.idea.quickfix.QuickFixTestGenerated.Libraries.testTestNG, ERROR: Couldn't find the following libraries,,
org.jetbrains.kotlin.idea.refactoring.move.MultiModuleMoveTestGenerated.testMoveFromJsModuleToJvmModule_MoveFromJsModuleToJvmModule, KT-34106,,
org.jetbrains.kotlin.idea.refactoring.move.MultiModuleMoveTestGenerated.testMoveFromJvmModuleToJsModule_MoveFromJvmModuleToJsModule, KT-34106,,
org.jetbrains.kotlin.idea.refactoring.move.MultiModuleMoveTestGenerated.testMoveJdkDependentToJsModule_MoveJdkDependentToJsModule, KT-34106,,
org.jetbrains.kotlin.idea.resolve.PartialBodyResolveTestGenerated.testSmartCastInTheSameStatement, KT-35922,,
@@ -56,14 +55,10 @@ org.jetbrains.kotlin.j2k.JavaToKotlinConverterSingleFileTestGenerated.AnonymousC
org.jetbrains.kotlin.j2k.JavaToKotlinConverterSingleFileTestGenerated.Issues.testKt_807,,,
org.jetbrains.kotlin.jps.build.IncrementalJvmJpsTestGenerated.WithJava.JavaUsedInKotlin.testMixedInheritance, Different clean up,,
org.jetbrains.kotlin.psi.injection.StringInterpolationInjectionTest.testInterpolationSimpleName, Stable failure during run may work in debug,,
org.jetbrains.kotlin.nj2k.NewJavaToKotlinConverterSingleFileTestGenerated.Issues.testKt_8170, KT-36459,,
org.jetbrains.kotlin.shortenRefs.ShortenRefsTestGenerated.testExtensionForObject2, Always red,,
org.jetbrains.kotlin.util.KotlinVersionsTest.testVersionsAreConsistent, KT-35567,,
org.jetbrains.kotlin.jvm.compiler.CompileKotlinAgainstJavaTestGenerated.WithAPT.testClassWithTypeParameter, KT-36448,,
org.jetbrains.kotlin.jvm.compiler.CompileKotlinAgainstJavaTestGenerated.WithoutAPT.testClassWithTypeParameter, KT-36448,,
org.jetbrains.kotlin.idea.debugger.test.AsyncStackTraceTestGenerated.testAllFilesPresentInAsyncStackTrace, redesign test AsyncStackTraces,,
org.jetbrains.kotlin.idea.debugger.test.AsyncStackTraceTestGenerated.testAsyncFunctions, redesign test AsyncStackTraces,,
org.jetbrains.kotlin.idea.debugger.test.AsyncStackTraceTestGenerated.testAsyncLambdas, redesign test AsyncStackTraces,,
org.jetbrains.kotlin.idea.debugger.test.AsyncStackTraceTestGenerated.testAsyncSimple, redesign test AsyncStackTraces,,
org.jetbrains.kotlin.idea.debugger.test.KotlinSteppingTestGenerated.StepOver.testStepOverInlinedLambdaStdlib, fails after advancing bootstrap KT-37879,,
org.jetbrains.kotlin.incremental.IncrementalJsKlibCompilerRunnerTestGenerated.ClassHierarchyAffected.testMethodRemoved, FO in klib required,,
1 Test key Issue State (optional: MUTE or FAIL) Status (optional: FLAKY)
13 org.jetbrains.kotlin.gradle.NewMultiplatformProjectImportingTest.testDetectAndroidSources[1: Gradle-4.9, KotlinGradlePlugin-latest stable] Gradle Import Tests
14 org.jetbrains.kotlin.gradle.NewMultiplatformProjectImportingTest.testDetectAndroidSources[3: Gradle-5.6.4, KotlinGradlePlugin-latest stable] Gradle Import Tests
15 org.jetbrains.kotlin.gradle.NewMultiplatformProjectImportingTest.testDetectAndroidSources[3: Gradle-5.6.4, KotlinGradlePlugin-latest stable] Gradle Import Tests
16 org.jetbrains.kotlin.gradle.NewMultiplatformProjectImportingTest.testSingleAndroidTarget[1: Gradle-4.9, KotlinGradlePlugin-latest stable] Stable on windows FLAKY
17 org.jetbrains.kotlin.gradle.NewMultiplatformProjectImportingTest.testSingleAndroidTarget[3: Gradle-5.6.4, KotlinGradlePlugin-latest stable] Stable on windows FLAKY
18 org.jetbrains.kotlin.idea.caches.resolve.MultiPlatformHighlightingTestGenerated.testJvmKotlinReferencesCommonKotlinThroughJavaDifferentJvmImpls Always red
19 org.jetbrains.kotlin.idea.codeInsight.gradle.GradleBuildFileHighlightingTest.testKtsInJsProject[1: with Gradle-5.6.4] ERROR: Diagnostic's list should be empty
20 org.jetbrains.kotlin.idea.codeInsight.gradle.GradleBuildFileHighlightingTest.testKtsInJsProject[0: with Gradle-4.9] ERROR: Diagnostic's list should be empty
21 org.jetbrains.kotlin.idea.codeInsight.gradle.GradleBuildFileHighlightingTest.testKtsInJsProject ERROR: Diagnostic's list should be empty
22 org.jetbrains.kotlin.idea.codeInsight.gradle.GradleConfiguratorTest.testAddLibraryGSK[1: with Gradle-5.6.4] unresolved kotlinModule
org.jetbrains.kotlin.idea.codeInsight.gradle.GradleConfiguratorTest.testAddNonKotlinLibraryGSK[1: with Gradle-5.6.4] unresolved kotlinModule
23 org.jetbrains.kotlin.idea.codeInsight.gradle.GradleConfiguratorTest.testAddTestLibraryGSK[1: with Gradle-5.6.4] unresolved kotlinModule
24 org.jetbrains.kotlin.idea.codeInsight.gradle.GradleConfiguratorTest.testConfigure10[1: with Gradle-5.6.4] absent test data
25 org.jetbrains.kotlin.idea.codeInsight.gradle.GradleConfiguratorTest.testConfigureGSK[1: with Gradle-5.6.4] absent test data
26 org.jetbrains.kotlin.idea.codeInsight.gradle.GradleConfiguratorTest.testConfigureGradleKtsKotlinDevVersion[1: with Gradle-5.6.4] absent test data
27 org.jetbrains.kotlin.idea.codeInsight.gradle.GradleConfiguratorTest.testConfigureKotlinDevVersion[1: with Gradle-5.6.4] absent test data
28 org.jetbrains.kotlin.idea.codeInsight.gradle.GradleConfiguratorTest.testConfigureKotlinWithPluginsBlock[1: with Gradle-5.6.4] absent test data
org.jetbrains.kotlin.idea.completion.test.JvmSmartCompletionTestGenerated.testFilterTo KT-32919 Broken in 71cb65c064c7 or 2d5a05466dfa
29 org.jetbrains.kotlin.idea.configuration.ConfigureKotlinInTempDirTest.testKotlincExistsNoSettingsLatestRuntimeNoVersionAutoAdvance FLAKY
30 org.jetbrains.kotlin.idea.configuration.ConfigureKotlinInTempDirTest.testNoKotlincExistsNoSettingsRuntime10 FLAKY
31 org.jetbrains.kotlin.idea.configuration.ConfigureKotlinTest.testJava9WithModuleInfo ERROR: Require directive for kotlin.stdlib is expected
32 org.jetbrains.kotlin.idea.configuration.GradleConfigureProjectByChangingFileTestGenerated.Gradle.testWithJava9ModuleInfo Probably test data should be updated
33 org.jetbrains.kotlin.idea.debugger.test.CoroutineDumpTestGenerated.testNoCoroutines Stable on windows FLAKY
34 org.jetbrains.kotlin.idea.debugger.test.CoroutineDumpTestGenerated.testThreeCoroutines Stable on windows FLAKY
35 org.jetbrains.kotlin.idea.debugger.test.CoroutineDumpTestGenerated.testTwoDumps Stable on windows FLAKY
36 org.jetbrains.kotlin.idea.decompiler.stubBuilder.BuiltInDecompilerConsistencyTest.testSameAsClsDecompilerForCompiledBuiltInClasses No classes with this name found: kotlin.ArithmeticException FLAKY
37 org.jetbrains.kotlin.idea.decompiler.stubBuilder.BuiltInDecompilerTest.testBuiltInStubTreeEqualToStubTreeFromDecompiledText No classes with this name found: kotlin.ArithmeticException FLAKY
38 org.jetbrains.kotlin.idea.highlighter.HighlightingTestGenerated.testSuspend Always red
40 org.jetbrains.kotlin.idea.maven.KotlinMavenInspectionTestGenerated.testWrongPhaseExecution Always red
41 org.jetbrains.kotlin.idea.maven.MavenMigrateTest.testMigrateApiAndLanguageVersions flaky on windows FLAKY
42 org.jetbrains.kotlin.idea.maven.configuration.MavenConfigureProjectByChangingFileTestGenerated.Jvm.testWithJava9ModuleInfo Probably test data should be updated
43 org.jetbrains.kotlin.idea.quickfix.DeprecatedSymbolUsageFixSpecialTest.testDefaultParameterValuesFromLibrary Always red because of additional import FLAKY
44 org.jetbrains.kotlin.idea.quickfix.QuickFixMultiModuleTestGenerated.CreateActual.testWithTestDummy Very flaky on teamcity FLAKY
45 org.jetbrains.kotlin.idea.quickfix.QuickFixTestGenerated.CreateFromUsage.CreateClass.CallExpression.testCallInWhenEntry KT-35728
46 org.jetbrains.kotlin.idea.quickfix.QuickFixTestGenerated.Libraries.testJunit ERROR: Couldn't find the following libraries
47 org.jetbrains.kotlin.idea.quickfix.QuickFixTestGenerated.Libraries.testTestNG ERROR: Couldn't find the following libraries
org.jetbrains.kotlin.idea.refactoring.move.MultiModuleMoveTestGenerated.testMoveFromJsModuleToJvmModule_MoveFromJsModuleToJvmModule KT-34106
48 org.jetbrains.kotlin.idea.refactoring.move.MultiModuleMoveTestGenerated.testMoveFromJvmModuleToJsModule_MoveFromJvmModuleToJsModule KT-34106
49 org.jetbrains.kotlin.idea.refactoring.move.MultiModuleMoveTestGenerated.testMoveJdkDependentToJsModule_MoveJdkDependentToJsModule KT-34106
50 org.jetbrains.kotlin.idea.resolve.PartialBodyResolveTestGenerated.testSmartCastInTheSameStatement KT-35922
55 org.jetbrains.kotlin.j2k.JavaToKotlinConverterSingleFileTestGenerated.Issues.testKt_807
56 org.jetbrains.kotlin.jps.build.IncrementalJvmJpsTestGenerated.WithJava.JavaUsedInKotlin.testMixedInheritance Different clean up
57 org.jetbrains.kotlin.psi.injection.StringInterpolationInjectionTest.testInterpolationSimpleName Stable failure during run may work in debug
org.jetbrains.kotlin.nj2k.NewJavaToKotlinConverterSingleFileTestGenerated.Issues.testKt_8170 KT-36459
58 org.jetbrains.kotlin.shortenRefs.ShortenRefsTestGenerated.testExtensionForObject2 Always red
59 org.jetbrains.kotlin.util.KotlinVersionsTest.testVersionsAreConsistent KT-35567
60 org.jetbrains.kotlin.jvm.compiler.CompileKotlinAgainstJavaTestGenerated.WithAPT.testClassWithTypeParameter KT-36448
61 org.jetbrains.kotlin.jvm.compiler.CompileKotlinAgainstJavaTestGenerated.WithoutAPT.testClassWithTypeParameter KT-36448
org.jetbrains.kotlin.idea.debugger.test.AsyncStackTraceTestGenerated.testAllFilesPresentInAsyncStackTrace redesign test AsyncStackTraces
org.jetbrains.kotlin.idea.debugger.test.AsyncStackTraceTestGenerated.testAsyncFunctions redesign test AsyncStackTraces
62 org.jetbrains.kotlin.idea.debugger.test.AsyncStackTraceTestGenerated.testAsyncLambdas redesign test AsyncStackTraces
org.jetbrains.kotlin.idea.debugger.test.AsyncStackTraceTestGenerated.testAsyncSimple redesign test AsyncStackTraces
63 org.jetbrains.kotlin.idea.debugger.test.KotlinSteppingTestGenerated.StepOver.testStepOverInlinedLambdaStdlib fails after advancing bootstrap KT-37879
64 org.jetbrains.kotlin.incremental.IncrementalJsKlibCompilerRunnerTestGenerated.ClassHierarchyAffected.testMethodRemoved FO in klib required