Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-12584

Vectorized join with partition column of type char does not trim spaces

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • 1.3.0, 2.0.0, 2.1.0
    • 1.3.0, 2.0.0
    • Vectorization
    • None

    Description

      When a table is partitioned on a column of type char and if join is performed on partitioned column then following exception gets thrown from hashtable loader

      Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.hive.serde2.SerDeException: Unexpected tag: 52 reserialized to 5
      	at org.apache.hadoop.hive.ql.exec.tez.ObjectCache.retrieve(ObjectCache.java:82)
      	at org.apache.hadoop.hive.ql.exec.tez.ObjectCache$1.call(ObjectCache.java:92)
      	... 4 more
      Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.hive.serde2.SerDeException: Unexpected tag: 52 reserialized to 5
      	at org.apache.hadoop.hive.ql.exec.tez.HashTableLoader.load(HashTableLoader.java:216)
      	at org.apache.hadoop.hive.ql.exec.MapJoinOperator.loadHashTable(MapJoinOperator.java:293)
      	at org.apache.hadoop.hive.ql.exec.MapJoinOperator$1.call(MapJoinOperator.java:174)
      	at org.apache.hadoop.hive.ql.exec.MapJoinOperator$1.call(MapJoinOperator.java:170)
      	at org.apache.hadoop.hive.ql.exec.tez.ObjectCache.retrieve(ObjectCache.java:75)
      	... 5 more
      Caused by: org.apache.hadoop.hive.serde2.SerDeException: Unexpected tag: 52 reserialized to 5
      	at org.apache.hadoop.hive.ql.exec.persistence.MapJoinBytesTableContainer$LazyBinaryKvWriter.sanityCheckKeyForTag(MapJoinBytesTableContainer.java:276)
      	at org.apache.hadoop.hive.ql.exec.persistence.MapJoinBytesTableContainer$LazyBinaryKvWriter.getHashFromKey(MapJoinBytesTableContainer.java:247)
      	at org.apache.hadoop.hive.ql.exec.persistence.HybridHashTableContainer.internalPutRow(HybridHashTableContainer.java:451)
      	at org.apache.hadoop.hive.ql.exec.persistence.HybridHashTableContainer.putRow(HybridHashTableContainer.java:444)
      	at org.apache.hadoop.hive.ql.exec.tez.HashTableLoader.load(HashTableLoader.java:210)
      

      Attachments

        1. HIVE-12584-branch-1.patch
          29 kB
          Prasanth Jayachandran
        2. HIVE-12584.2.patch
          41 kB
          Prasanth Jayachandran
        3. HIVE-12584.1.patch
          41 kB
          Prasanth Jayachandran

        Activity

          People

            prasanth_j Prasanth Jayachandran
            jvaria Jagruti Varia
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: