ParquetWriter write method is taking too long to write single avro message #2133

Activity

1. HenryCaiHaiying commented on Jul 30, 2021

   

   HenryCaiHaiying

   on Jul 30, 2021 via email

   Contributor

   More actions

   You can do a few thread dump to see what the process stuck at. Adding memory probably will help parquet conversion.

   …

   On Tue, Jul 27, 2021 at 1:52 AM aksamit26 ***@***.***> wrote: When i am reading data form Kafka , i am deserialising it to Avro record. But ParquetWriter write method is taking around 2~5 secs to write one single avro record. Is there any such issue surfaced earlier. What can be possible solve for this ? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#2133>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABYJP74TKFG6EMN5TRTVTJDTZZXVVANCNFSM5BBYSN7A> .