2
\$\begingroup\$

I am working on a Spring-MVC application in which we have note like functionality. Notes can have attachments, history, etc.
There is a method to copy notes. When a note has a large number of attachments (big or small), it takes a heck load of time for it to copy (around 5 minutes for a note with 341 attachments).

Yesterday I posted a question about the same here, and the advice to flush the session and clear the cache same as batch size helped. The method at least works now. But 5 minutes is unacceptable.

I would like to know what am I doing wrong, and how I can improve it. In the method, you will see that I am copying the properties of objects mańually. When I use Bean copy functionality, it throws me errors for its dependent objects(one-to-many).

The code is long and I am posting the config as well to be thorough. Note: I am skipping the Google Drive and Dropbox code, as it's not used that much and is a lower priority. The method addAttachmentForCopy() is taking a lot of time.

root-context.xml:

 <beans:bean id="dataSource" class="com.zaxxer.hikari.HikariDataSource"  destroy-method="close">
    <beans:property name="dataSourceClassName" value="org.postgresql.ds.PGSimpleDataSource"/>

   <beans:property name="maximumPoolSize" value="100" />
    <beans:property name="maxLifetime" value="300000" />
    <beans:property name="idleTimeout" value="300000" />
    <beans:property name="dataSourceProperties">
        <beans:props>
            <beans:prop key="url">jdbc:postgresql://localhost:PORT_NOS/DB_NAME</beans:prop>
            <beans:prop key="user">USERNAME</beans:prop>
            <beans:prop key="password">PASSWORD</beans:prop>

        </beans:props>
    </beans:property>
</beans:bean>

<!-- Hibernate 4 SessionFactory Bean definition -->
<beans:bean id="hibernate4AnnotatedSessionFactory"
            class="org.springframework.orm.hibernate4.LocalSessionFactoryBean">
    <beans:property name="dataSource" ref="dataSource"/>
    <beans:property name="packagesToScan" value="com.ourapp.spring.model"/>
    <beans:property name="hibernateProperties">
        <beans:props>
            <beans:prop key="hibernate.dialect">org.hibernate.dialect.PostgreSQL9Dialect</beans:prop>
            <beans:prop key="hibernate.show_sql">false</beans:prop>
            <beans:prop key="hibernate.jdbc.batch_size">50</beans:prop>
            <beans:prop key="hibernate.hbm2ddl.auto">update</beans:prop>
            <beans:prop key="cache.use_second_level_cache">true</beans:prop>
            <beans:prop key="cache.use_query_cache">true</beans:prop>
            <beans:prop key="hibernate.order_updates">true</beans:prop>
            <beans:prop key="show_sql">false</beans:prop>
        </beans:props>
    </beans:property>

</beans:bean>

GroupNotesServiceImpl duplicateNote:

@Override
public int duplicateGroupNote(int groupNoteId, int sectionId, long groupAccountId, boolean duplicateFlag,
                              boolean sendNotification, double sortValue, boolean copyFlag) {
    Person person = this.personService.getCurrentlyAuthenticatedUser();
    GroupNotes savedNote = this.groupNotesDAO.getGroupNoteById(groupNoteId);
    GroupSection originSection = this.groupSectionService.getGroupSectionById(savedNote.getOwnedSectionId());
    GroupCanvas originCanvas = this.groupCanvasService.getCanvasById(originSection.getCurrentCanvasId());
    GroupAccount originAccount = this.groupAccountService.getGroupById(originCanvas.getGroupAccountId());
    GroupMembers etherPadMember = this.groupMembersService.returnMembersMatchingUsernameAccountId(person.getUsername(), groupAccountId);
    GroupAccount destinationGroupAccount = this.groupAccountService.getGroupById(groupAccountId);
    GroupSection groupSection = this.groupSectionService.getGroupSectionById(sectionId);
    GroupCanvas groupCanvas = this.groupCanvasService.getCanvasById(groupSection.getCurrentCanvasId());
    GroupAccount groupAccount = this.groupAccountService.getGroupById(groupCanvas.getGroupAccountId());
    Permission permission = this.permissionService.getPermissionForMember(groupAccount.getGroupId());
    GroupMembers actionMember = this.groupMembersService.returnMembersMatchingUsernameAccountId(person.getUsername(), originAccount.getGroupId());

    String teamAndBoardNameOrigin = originAccount.getGroupName() + ", " + originCanvas.getMcanvasname();
    String teamAndBoardTextOrigin = savedNote.getMnotetag() + ", Note Verschoben";

    String teamAndBoardNameDestination = groupAccount.getGroupName() + ", " + groupCanvas.getMcanvasname();

    List<Object> toSendObjectList = new ArrayList<>();
    final int[] updatedNoteId = new int[1];
    if (permission.isAddNote()) {
        this.activityCounterService.saveGroupNotActivity(groupNoteId);
        this.noteActivityService.saveGroupNotActivity(actionMember.getMemberid());
        GroupNotes duplicateNote;
        if (duplicateFlag) {
            duplicateNote = new GroupNotes("Kopie von: " + savedNote.getMnotetag(), "",
                    sortValue, savedNote.getMnotecolor(),
                    new Timestamp(System.currentTimeMillis()), savedNote.getCreatorId(),
                    savedNote.getNoteCreatorEmail(), savedNote.getNoteDate());
        } else {
            duplicateNote = new GroupNotes(savedNote.getMnotetag(), "",
                    sortValue, savedNote.getMnotecolor(),
                    new Timestamp(System.currentTimeMillis()), savedNote.getCreatorId(),
                    savedNote.getNoteCreatorEmail(), savedNote.getNoteDate());
        }
        duplicateNote.setAttachCount(savedNote.getAttachCount());
        duplicateNote.setCanvasUrl(savedNote.getCanvasUrl());
        duplicateNote.setCollapse(savedNote.isCollapse());
        duplicateNote.setDisabledSortId(savedNote.getDisabledSortId());
        duplicateNote.setEndIsMilestone(savedNote.isEndIsMilestone());
        duplicateNote.setGannt(false);
        duplicateNote.setGanntOrder(0);
        duplicateNote.setGanttDuration(0);
        duplicateNote.setGanttStatus("");
        duplicateNote.setRandomNos(UUID.randomUUID().toString());
        duplicateNote.setLevel(0);
        duplicateNote.setNoteDisabled(savedNote.isNoteDisabled());
        duplicateNote.setNoteInActive(savedNote.isNoteInActive());
        duplicateNote.setNoteTarget(savedNote.getNoteTarget());
        duplicateNote.setUploader(savedNote.getUploader());
        duplicateNote.setCanvasId(groupCanvas.getMcanvasid());
        duplicateNote.setGroupId(groupAccount.getGroupId());
        duplicateNote.setUrgent(savedNote.isUrgent());
        duplicateNote.setTags(savedNote.getTags());
        duplicateNote.setErledigtPersonId(savedNote.getErledigtPersonId());
        duplicateNote.setErledigtTimestamp(savedNote.getErledigtTimestamp());
        duplicateNote.setFinished(savedNote.isFinished());
        duplicateNote.setProcessing(true);

        try {
            if ((savedNote.getPadId() != null) && (!savedNote.getPadId().isEmpty())) {
                String text = this.initailizeEpadClient.getTextForGivePad(savedNote.getPadId());
                //text = Jsoup.parse(text).text();
                String padID = this.initailizeEpadClient.createPadObjectsWithHtmlForDuplicateNotes(etherPadMember.getSessionData(), text);

                duplicateNote.setPadId(padID);
                //this.initailizeEpadClient.copyPad(savedNote.getPadId(), padID);
                String oldText = this.initailizeEpadClient.getTextForGivePad(padID);
                if (oldText != null) {
                    duplicateNote.setMnotetext(oldText);
                }
            } else {
                duplicateNote.setMnotetext(savedNote.getMnotetext());
            }
        } catch (Exception e) {
            e.printStackTrace();

            duplicateNote.setMnotetext(savedNote.getMnotetext());
        }

        if (!duplicateFlag) {
            duplicateNote.setNoteNumber(savedNote.getNoteNumber());
            duplicateNote.setCreatorId(savedNote.getCreatorId());
        } else {
            duplicateNote.setCreatorId(person.getId());
            this.groupCanvasService.directUpdateGroupCanvas(groupCanvas);
            groupCanvas = this.groupCanvasService.getCanvasById(groupCanvas.getMcanvasid());
            groupCanvas.setNoteCount(groupCanvas.getNoteCount() + 1);
            this.groupCanvasService.directUpdateGroupCanvas(groupCanvas);
            duplicateNote.setNoteNumber(groupCanvas.getNoteCount());
        }
        duplicateNote.setFinished(savedNote.isFinished());
        duplicateNote.setExcelData(savedNote.getExcelData());
        if (savedNote.getZugwisenPersonId() != 0) {
            Person zugweisedUser = this.personService.getPersonById(savedNote.getZugwisenPersonId());
            if (zugweisedUser != null) {
                GroupMembers groupMembers = this.groupMembersService.returnMembersMatchingUsernameAccountId(zugweisedUser.getUsername(), groupAccountId);
                if (groupMembers != null) {
                    duplicateNote.setZugwisenPersonId(savedNote.getZugwisenPersonId());
                    duplicateNote.setZugwisenPersonTimestamp(savedNote.getZugwisenPersonTimestamp());
                    duplicateNote.setPersonWhoZugweised(savedNote.getPersonWhoZugweised());
                } else {
                    duplicateNote.setZugwisenPersonId(0);
                    duplicateNote.setZugwisenPersonTimestamp(null);
                    duplicateNote.setPersonWhoZugweised(0);
                }
            }
        }
        duplicateNote.setStartDateTimestamp(savedNote.getStartDateTimestamp());
        duplicateNote.setPrefix(savedNote.getPrefix());
        duplicateNote.setTaskRelations(savedNote.getTaskRelations());
        duplicateNote.setPrivateNoteFlag(savedNote.isPrivateNoteFlag());
        duplicateNote.setProcessing(true);
        try {
            if (duplicateFlag) {
                updatedNoteId[0] = this.groupNotesDAO.saveGroupNoteAndReturnId(duplicateNote, sectionId);
            } else {
                updatedNoteId[0] = this.groupNotesDAO.saveGroupNoteAndReturnId(duplicateNote, sectionId);
                if (originAccount.getGroupId() != groupAccount.getGroupId()) {
                    this.activityCounterService.saveGroupNotActivity(updatedNoteId[0]);
                    GroupMembers groupMembers = this.groupMembersService.returnMembersMatchingUsernameAccountId(person.getUsername(), groupAccount.getGroupId());
                    if (groupMembers != null) {
                        this.noteActivityService.saveGroupNotActivity(groupMembers.getMemberid());
                    }
                }
            }
            sendControlNotificationsForNote(null, updatedNoteId[0], groupSection.getMsectionid(), groupCanvas.getMcanvasid(), destinationGroupAccount.getGroupId(), false);
            GroupNotes databaseNoteObject = this.groupNotesDAO.getGroupNoteById(updatedNoteId[0]);
            this.groupAttachmentsService.saveIndexes(databaseNoteObject.getMnotetext() + " " + databaseNoteObject.getTags(), databaseNoteObject.getMnotetag(), null, groupAccount.getGroupId(), true, databaseNoteObject.getMnoticesid());
            List<Integer> noteFollowers = this.noteFollowService.getNoteFollowsForNote(groupNoteId);
            if (noteFollowers != null) {
                for (Integer id : noteFollowers) {
                    this.noteFollowService.addNoteFollow(id, updatedNoteId[0]);
                }
            }
        } catch (Exception e) {
            e.printStackTrace();

        }

        List<GroupNoteHistory> groupNoteHistoryList = this.groupNoteHistoryService.getNoteHistoryById(savedNote.getMnoticesid());
        try {
            int i = 0;
            for (GroupNoteHistory groupNoteHistory : groupNoteHistoryList) {
                i++;
                GroupNoteHistory targetGroupHistory = new GroupNoteHistory();
                targetGroupHistory.setChangedMessage(groupNoteHistory.getChangedMessage());
                targetGroupHistory.setEditorId(groupNoteHistory.getEditorId());
                targetGroupHistory.setFirstName(groupNoteHistory.getFirstName());
                targetGroupHistory.setMnoteEditDate(groupNoteHistory.getMnoteEditDate());
                targetGroupHistory.setNestedComment(groupNoteHistory.isNestedComment());
                targetGroupHistory.setNestedId(groupNoteHistory.getNestedId());
                targetGroupHistory.setNestedCommentList(groupNoteHistory.getNestedCommentList());
                targetGroupHistory.setNewNoteHeadline(groupNoteHistory.getNewNoteHeadline());
                targetGroupHistory.setNewNoteText(groupNoteHistory.getNewNoteText());
                targetGroupHistory.setOldHeadLine(groupNoteHistory.getOldHeadLine());
                targetGroupHistory.setOldText(groupNoteHistory.getOldText());
                targetGroupHistory.setParent(groupNoteHistory.isParent());
                targetGroupHistory.setPinned(groupNoteHistory.isPinned());
                targetGroupHistory.setPinnedTime(groupNoteHistory.getPinnedTime());
                targetGroupHistory.setWhatHasChanged(groupNoteHistory.getWhatHasChanged());

                int saveId = this.groupNoteHistoryService.addNoteHistory(targetGroupHistory, updatedNoteId[0]);
                if (saveId != 0) {
                    toSendObjectList.add(this.groupNoteHistoryService.getGroupNoteHistoryById(saveId));
                }
                if (groupNoteHistory.isParent()) {
                    int j = 0;
                    List<GroupNoteHistory> nestedList = this.groupNoteHistoryService.getChildHistoryEntries(groupNoteHistory.getMhistoryid());
                    for (GroupNoteHistory nestedHistory : nestedList) {
                        j++;
                        GroupNoteHistory nestedHistoryEntry = new GroupNoteHistory();
                        nestedHistoryEntry.setFirstName(nestedHistory.getFirstName());
                        nestedHistoryEntry.setChangedMessage(nestedHistory.getChangedMessage());
                        nestedHistoryEntry.setEditorId(nestedHistory.getEditorId());
                        nestedHistoryEntry.setFirstName(nestedHistory.getFirstName());
                        nestedHistoryEntry.setMnoteEditDate(nestedHistory.getMnoteEditDate());
                        nestedHistoryEntry.setNestedComment(nestedHistory.isNestedComment());
                        nestedHistoryEntry.setNestedCommentList(nestedHistory.getNestedCommentList());
                        nestedHistoryEntry.setNewNoteHeadline(nestedHistory.getNewNoteHeadline());
                        nestedHistoryEntry.setNewNoteText(nestedHistory.getNewNoteText());
                        nestedHistoryEntry.setOldHeadLine(nestedHistory.getOldHeadLine());
                        nestedHistoryEntry.setOldText(nestedHistory.getOldText());
                        nestedHistoryEntry.setParent(nestedHistory.isParent());
                        nestedHistoryEntry.setPinned(nestedHistory.isPinned());
                        nestedHistoryEntry.setPinnedTime(nestedHistory.getPinnedTime());
                        nestedHistoryEntry.setWhatHasChanged(nestedHistory.getWhatHasChanged());
                        nestedHistoryEntry.setNestedId(saveId);
                        int nestedId = this.groupNoteHistoryService.addNoteHistory(nestedHistoryEntry, updatedNoteId[0]);
                        if (nestedId != 0) {
                            toSendObjectList.add(this.groupNoteHistoryService.getGroupNoteHistoryById(nestedId));
                        }
                        if (j % 50 == 0) {
                            this.groupNoteHistoryService.flushSession();
                        }

                    }
                }
                if (i % 50 == 0) {
                    this.groupNoteHistoryService.flushSession();
                }

            }
        } catch (Exception e) {
          e.printStackTrace();

        }
        this.groupNoteHistoryService.flushSession();
        GroupNotes dbNote = getGroupNoteObjectById(updatedNoteId[0]);
        try {
            try {
                List<GroupAttachments> groupAttachmentsSet = this.groupAttachmentsService.getAttachmenByNoteId(groupNoteId);
                Collections.reverse(groupAttachmentsSet);
                boolean flag;
                int i = 0;
                for (GroupAttachments groupAttachments : groupAttachmentsSet) {
                    flag = false;
                    i++;
                    if ((groupAttachments.getApiUsed() == null) || (groupAttachments.getApiUsed().isEmpty())) {
                        flag = true;
                        GroupAttachments attachmentWithData = this.groupAttachmentsService
                                .getAttachmenById(groupAttachments.getGroupAttachId());
                        if (attachmentWithData != null) {
                                this.groupAttachmentsService.addAttachmentForCopy(attachmentWithData, dbNote, false,
                                        groupSection, groupCanvas, destinationGroupAccount);
                        }
                    }


                    if (!(groupAttachments.getApiUsed() == null)) {
                        if (!(groupAttachments.getApiUsed().isEmpty())) {
                            if ((groupAttachments.getApiUsed().equals("google")) && (!flag)) {
                                flag = true;

                                if (destinationGroupAccount.isUseDrive()) {
                                    InputStream inputStream = this.driveQuickstart.downloadFile(groupAttachments
                                            .getGroupAttachId(), destinationGroupAccount.getGroupId(), false);
                                    byte[] bytes = IOUtils.toByteArray(inputStream);

                                    this.driveQuickstart.insertFile(bytes, groupAttachments.getFileName(),
                                            groupAttachments.getMimeType(), groupAttachments.getFileSize(),
                                            updatedNoteId[0], destinationGroupAccount.getGroupId(),
                                            "", false, groupAttachments);

                                }
                            }
                        }
                    }

                    if (!(groupAttachments.getApiUsed() == null)) {
                        if (!(groupAttachments.getApiUsed().isEmpty())) {
                            if (((groupAttachments.getApiUsed().equals("dropbox")) && (!flag))) {
                                flag = true;

                                if (destinationGroupAccount.isUseDropbox()) {
                                    InputStream inputStream = this.dropboxTask.downloadFile(groupAttachments
                                            .getGroupAttachId()
                                            , false);
                                    byte[] bytes = IOUtils.toByteArray(inputStream);

                                    this.dropboxTask.insertFileAsByteArray(bytes, groupAttachments.getFileName(),
                                            groupAttachments.getMimeType(),
                                            groupAttachments.getFileSize(), "", updatedNoteId[0],
                                            destinationGroupAccount.getGroupId(), false, groupAttachments);
                                }
                            }
                        }
                    }

                    if (!flag) {
                        if (!(groupAttachments.getApiUsed() == null)) {
                            if (!(groupAttachments.getApiUsed().isEmpty())) {
                                if ((groupAttachments.getApiUsed().equals("google"))) {
                                    flag = true;
                                    InputStream inputStream = this.driveQuickstart.downloadFile(groupAttachments
                                            .getGroupAttachId(), destinationGroupAccount.getGroupId(), true);

                                    byte[] bytes = IOUtils.toByteArray(inputStream);
                                    groupAttachments.setSendAttachment(bytes);
                                    this.groupAttachmentsService.addAttachmentForCopy(groupAttachments, dbNote, false,
                                            groupSection, groupCanvas, destinationGroupAccount);
                                }
                            }
                        }

                        if (!(groupAttachments.getApiUsed() == null)) {
                            if (!(groupAttachments.getApiUsed().isEmpty())) {
                                if (groupAttachments.getApiUsed().equals("dropbox")) {
                                    flag = true;


                                    InputStream inputStream = this.dropboxTask.downloadFile(groupAttachments.getGroupAttachId()
                                            , false);
                                    byte[] bytes = IOUtils.toByteArray(inputStream);

                                    groupAttachments.setSendAttachment(bytes);
                                    this.groupAttachmentsService.addAttachmentForCopy(groupAttachments, dbNote, false,
                                            groupSection, groupCanvas, destinationGroupAccount);
                                }
                            }
                        }

                        if (!flag) {
                            GroupAttachments attachmentWithData = this.groupAttachmentsService
                                    .getAttachmenById(groupAttachments.getGroupAttachId());
                            if (attachmentWithData != null) {
                                this.groupAttachmentsService.addAttachmentForCopy(attachmentWithData, dbNote, false,
                                        groupSection, groupCanvas, destinationGroupAccount);
                            }
                        }

                    }
                    if (i % 50 == 0) {
                        this.groupNoteHistoryService.flushSession();
                    }

                }
            } catch (Exception e) {
                 e.printStackTrace();

            }

        } catch (Exception e) {
            e.printStackTrace();

        }
        this.groupNoteHistoryService.flushSession();


        if ((!duplicateFlag)) {
            final GroupCanvas finalGroupCanvas = groupCanvas;

            Thread saveThread = new Thread(() -> {
                List<GroupMembers> groupMembersList = this.groupMembersService.returnGroupMembers(groupAccount.getGroupId());
                boolean archiveSuccess = this.groupNotesDAO.setNoteStatus(groupNoteId, true);
                if (archiveSuccess && sendNotification) {
                    for (GroupMembers groupMembers : groupMembersList) {
                        if (!(person.getUsername().equals(groupMembers.getMemberUsername()))) {
                            NotificationType notificationType = this.notificationTypeService.getNotificationTypeForMember(groupMembers.getMemberid());
                            if (notificationType.isRemoveNote()) {
                                Notification notification = this.notificationService.createNotification(person.getId(), false, "",
                                        groupAccount.getGroupName() + ", " + finalGroupCanvas.getMcanvasname(),
                                        "", "", originSection.getMsectionid(),
                                        originCanvas.getMcanvasid(), savedNote.getMnoticesid(), savedNote.getMnotecolor(), groupAccountId);
                                notification.setBroadCast(false);
                                notification.setZugweisedPersonId(savedNote.getZugwisenPersonId());
                                notification.setSendNoteId(groupNoteId);
                                notification.setRemove(true);
                                notification.setSubject(teamAndBoardNameOrigin);
                                notification.setText(teamAndBoardTextOrigin);
                                this.notificationService.addNotification(notification,
                                        groupMembers.getMemberid());
                            }

                            if (groupMembers.isNotifyWhenNoteDeleted()) {
                                if (!(person.getUsername().equals(groupMembers.getMemberUsername()))) {
                                    Person memberPerson = this.personService.findPersonByUsername(groupMembers.getMemberUsername());
                                    Permission emailPerm = this.permissionService.getPermissionByEmailAndGroupId(groupMembers.getTempgroupaccountid(), memberPerson.getId());
                                    if (emailPerm != null) {
                                        if ((emailPerm.isAllNotes()) ||
                                                (savedNote.getZugwisenPersonId() == groupMembers.getTransientPerson().getId())) {
                                            boolean allowed = this.canvasAllowedService.checkIfCanvasAllowedForMember(groupMembers.getMemberid(), originCanvas.getMcanvasid());
                                            if (allowed) {
                                                sendNoteDeletionEmail(originAccount.getGroupName(),
                                                        groupMembers.getMemberUsername(), person.getFirstName(),
                                                        savedNote.getMnotetag(), originCanvas.getMcanvasname(), originSection.getMsectionname(),
                                                        originCanvas.getMcanvasid(),
                                                        savedNote, originAccount.getGroupId(), person.getId());
                                            }
                                        }
                                    }
                                }
                            }
                        } else {
                            sendControlNotificationsForNote(null, groupNoteId, originSection.getMsectionid(), originCanvas.getMcanvasid(), originAccount.getGroupId(), true);
                        }
                    }
                } else {
                    sendControlNotificationsForNote(null, groupNoteId, originSection.getMsectionid(), originCanvas.getMcanvasid(), originAccount.getGroupId(), true);
                }
            });
            saveThread.start();
        }

        GroupNotes updatedNote =
                this.groupNotesDAO.getGroupNoteById(updatedNoteId[0]);
        if (!duplicateFlag) {
            int copyHistoryID = this.groupNoteHistoryService.addNoteHistory(new GroupNoteHistory(new Timestamp(System.currentTimeMillis()),
                    "cut", "Diese Note wurde in das Board " + groupCanvas.getMcanvasname() + " ausgeschnitten",
                    person.getId()), groupNoteId);
            GroupNoteHistory copyHistory = this.groupNoteHistoryService.getGroupNoteHistoryById(copyHistoryID);
            toSendObjectList.add(copyHistory);

            List<GroupMembers> groupMembersList = this.groupMembersService.returnGroupMembers(originAccount.getGroupId());
            for (GroupMembers groupMembers : groupMembersList) {
                if (!person.getUsername().equals(groupMembers.getMemberUsername())) {
                    NotificationType notificationType = this.notificationTypeService.getNotificationTypeForMember(groupMembers.getMemberid());
                    if (notificationType.isNoteCopy()) {
                        Notification copyNotification = new Notification(teamAndBoardNameOrigin + "Note Ausgeschnitten", "Note Ausgeschnitten",
                                originSection.getMsectionid(), originCanvas.getMcanvasid(),
                                person.getId(), false, new Timestamp(System.currentTimeMillis()), "copy",
                                savedNote.getMnotecolor(), true, originAccount.getGroupId());
                        copyNotification.setSendNoteId(groupNoteId);
                        copyNotification.setObjects(toSendObjectList);
                        copyNotification.setSubject(teamAndBoardNameOrigin);
                        copyNotification.setText(teamAndBoardTextOrigin);
                        this.chatService.sendNotification(copyNotification, groupMembers.getMemberid());
                    }
                }
            }

            // toSendObjectList.clear();


            int pasteId = this.groupNoteHistoryService.addNoteHistory(new GroupNoteHistory(new Timestamp(System.currentTimeMillis()),
                    "paste", "Eingefügt in: " + groupAccount.getGroupName() + ", " + groupCanvas.getMcanvasname() + ", " + groupSection.getMsectionname(),
                    person.getId()), updatedNote.getMnoticesid());
            GroupNoteHistory pasteHistory = this.groupNoteHistoryService.getGroupNoteHistoryById(pasteId);
            toSendObjectList.add(pasteHistory);

            List<GroupMembers> updatedGroupMembersList = this.groupMembersService.returnGroupMembers(groupAccountId);
            for (GroupMembers groupMembers : updatedGroupMembersList) {
                if (!person.getUsername().equals(groupMembers.getMemberUsername())) {
                    Notification pasteNotification = new Notification(teamAndBoardNameDestination + "Note Eingefügt", "Note Eingefügt",
                            groupSection.getMsectionid(), groupCanvas.getMcanvasid(),
                            person.getId(), false, new Timestamp(System.currentTimeMillis()), "copy",
                            savedNote.getMnotecolor(), true, groupAccount.getGroupId());
                    pasteNotification.setSendNoteId(updatedNoteId[0]);
                    pasteNotification.setObjects(toSendObjectList);
                    pasteNotification.setSubject(teamAndBoardNameDestination);
                    pasteNotification.setText(updatedNote.getMnotetag() + ", Note Eingefügt");
                    this.chatService.sendNotification(pasteNotification, groupMembers.getMemberid());

                }
            }


        } else {
            if (!copyFlag) {
                int originCopyId = this.groupNoteHistoryService.addNoteHistory(new GroupNoteHistory(new Timestamp(System.currentTimeMillis()),
                        "copy", "Diese Note wurde dupliziert",
                        person.getId()), groupNoteId);
                List<Object> objects = new ArrayList<>();
                objects.add(this.groupNoteHistoryService.getGroupNoteHistoryById(originCopyId));
                sendControlNotificationsForNote(objects, groupNoteId, originSection.getMsectionid(), originCanvas.getMcanvasid(), originAccount.getGroupId(), false);
            } else {
                int originCopyId = this.groupNoteHistoryService.addNoteHistory(new GroupNoteHistory(new Timestamp(System.currentTimeMillis()),
                        "copy", "Diese Note wurde kopiert",
                        person.getId()), groupNoteId);
                List<Object> objects = new ArrayList<>();
                objects.add(this.groupNoteHistoryService.getGroupNoteHistoryById(originCopyId));
                sendControlNotificationsForNote(objects, groupNoteId, originSection.getMsectionid(), originCanvas.getMcanvasid(), originAccount.getGroupId(), false);
            }


            int copyHistoryID = this.groupNoteHistoryService.addNoteHistory(new GroupNoteHistory(new Timestamp(System.currentTimeMillis()),
                    "copy", "Kopiert von: " + originAccount.getGroupName() + ", " + originCanvas.getMcanvasname() + ", " + originSection.getMsectionname(),
                    person.getId()), updatedNote.getMnoticesid());

            GroupNoteHistory copyHistory = this.groupNoteHistoryService.getGroupNoteHistoryById(copyHistoryID);
            toSendObjectList.add(copyHistory);
            if (!sendNotification) {
                List<GroupMembers> groupMembersList = this.groupMembersService.returnGroupMembers(originAccount.getGroupId());
                for (GroupMembers groupMembers : groupMembersList) {
                    if (!person.getUsername().equals(groupMembers.getMemberUsername())) {
                        NotificationType notificationType = this.notificationTypeService.getNotificationTypeForMember(groupMembers.getMemberid());
                        if (notificationType.isNoteCopy()) {
                            Notification copyNotification = new Notification(teamAndBoardNameOrigin + "Note Kopiert", "Note kopiert",
                                    originSection.getMsectionid(), originCanvas.getMcanvasid(),
                                    person.getId(), false, new Timestamp(System.currentTimeMillis()), "copy",
                                    savedNote.getMnotecolor(), true, originAccount.getGroupId());
                            copyNotification.setSendNoteId(updatedNoteId[0]);
                            copyNotification.setObjects(toSendObjectList);
                            copyNotification.setSubject(teamAndBoardNameOrigin);
                            copyNotification.setText(teamAndBoardTextOrigin);
                            this.chatService.sendNotification(copyNotification, groupMembers.getMemberid());
                        }
                    }
                }
            }


            int pasteId = this.groupNoteHistoryService.addNoteHistory(new GroupNoteHistory(new Timestamp(System.currentTimeMillis()),
                    "paste", "Eingefügt in: " + groupAccount.getGroupName() + ", " + groupCanvas.getMcanvasname() + ", " + groupSection.getMsectionname(),
                    person.getId()), updatedNote.getMnoticesid());

            GroupNoteHistory pasteHistory = this.groupNoteHistoryService.getGroupNoteHistoryById(pasteId);
            toSendObjectList.add(pasteHistory);
            if (!sendNotification) {
                List<GroupMembers> updatedGroupMembersList = this.groupMembersService.returnGroupMembers(groupAccountId);
                for (GroupMembers groupMembers : updatedGroupMembersList) {
                    if (!person.getUsername().equals(groupMembers.getMemberUsername())) {
                        Notification pasteNotification = new Notification("Note Eingefügt: " + savedNote.getMnotetag(), "Note kopiert",
                                originSection.getMsectionid(), originCanvas.getMcanvasid(),
                                person.getId(), false, new Timestamp(System.currentTimeMillis()), "copy",
                                savedNote.getMnotecolor(), true, originAccount.getGroupId());
                        pasteNotification.setSendNoteId(updatedNoteId[0]);
                        pasteNotification.setObjects(toSendObjectList);
                        this.chatService.sendNotification(pasteNotification, groupMembers.getMemberid());

                    }
                }
            }
        }


        GroupSection updatedSection = this.groupSectionService.getGroupSectionById(updatedNote.getOwnedSectionId());
        GroupCanvas updatedCanvas = this.groupCanvasService.getCanvasById(updatedSection.getCurrentCanvasId());
        final GroupCanvas finalGroupCanvas1 = groupCanvas;
        if (sendNotification) {
            List<GroupMembers> newMemberList = this.groupMembersService.returnGroupMembers(groupAccountId);
            for (GroupMembers members : newMemberList) {
                if (members.getMemberUsername().equals(person.getUsername())) {
                    sendControlNotificationsForNote(toSendObjectList, updatedNoteId[0], groupSection.getMsectionid(), finalGroupCanvas1.getMcanvasid(), destinationGroupAccount.getGroupId(), false);
                } else {
                    NotificationType notificationType = this.notificationTypeService.getNotificationTypeForMember(members.getMemberid());
                    if (notificationType.isAddNote()) {
                        Notification notification = this.notificationService.createNotification(person.getId(), true, "/section/listing/" +
                                        updatedCanvas.getMcanvasid() + "?note="
                                        + updatedNote.getMnoticesid(),
                                groupAccount.getGroupName() + ", " + updatedCanvas.getMcanvasname(), "", "note", updatedSection.getMsectionid(),
                                updatedCanvas.getMcanvasid(), 0, updatedNote.getMnotecolor(), destinationGroupAccount.getGroupId());
                        notification.setBroadCast(false);
                        notification.setZugweisedPersonId(updatedNote.getZugwisenPersonId());
                        notification.setNoteid(updatedNoteId[0]);
                        notification.setSendNoteId(updatedNoteId[0]);
                        notification.setObjects(toSendObjectList);
                        if (duplicateFlag) {
                            notification.setText("Note kopiert/ eingefügt: <span class='bold'>" + updatedNote.getMnotetag() + "</span> ");
                        } else {
                            notification.setText("Note ausgeschn./ eingefügt: <span class='bold'>" + updatedNote.getMnotetag() + "</span> ");
                        }
                        this.chatService.sendNotification(notification, members.getMemberid());
                        this.notificationService.addNotification(notification, members.getMemberid());
                    } else {
                        sendControlNotificationsForNote(toSendObjectList, updatedNoteId[0], groupSection.getMsectionid(), finalGroupCanvas1.getMcanvasid(), destinationGroupAccount.getGroupId(), false);
                    }
                }

            }
        } else {
            sendControlNotificationsForNote(toSendObjectList, updatedNoteId[0], groupSection.getMsectionid(), finalGroupCanvas1.getMcanvasid(), destinationGroupAccount.getGroupId(), false);
        }
        GroupNotes processNote = this.groupNotesDAO.getGroupNoteById(updatedNoteId[0]);
        if (processNote != null) {
            processNote.setProcessing(false);
            processNote.setFinished(savedNote.isFinished());
            this.groupNotesDAO.editGroupNote(processNote, sectionId);
            List<GroupAttachments> groupAttachmentsList = this.groupAttachmentsService.getAttachmenByNoteId(updatedNoteId[0]);
            List<Object> objectList = new ArrayList<>();
            objectList.addAll(groupAttachmentsList);
            sendControlNotificationsForNote(objectList, updatedNoteId[0], groupSection.getMsectionid(), groupCanvas.getMcanvasid(), destinationGroupAccount.getGroupId(), false);
        }
        return updatedNoteId[0];
    }

    return 0;
}

Method taking a lot of time addAttachmentForCopy() :

@Override
public boolean addAttachmentForCopy(GroupAttachments originalFile, GroupNotes groupNotes,
                                    boolean notify, GroupSection groupSection,
                                    GroupCanvas groupCanvas, GroupAccount groupAccount){
    try {
        if (!(originalFile.getSendAttachment() == null)) {
            Person person = this.personService.getCurrentlyAuthenticatedUser();
            Permission permission = this.permissionService.getPermissionForMember(groupAccount.getGroupId());
            boolean uploadAllowed = this.personService.isUploadAllowed(originalFile.getFileSize(), person.getUsername());
            Person attachmentUploader = this.personService.getPersonById(originalFile.getUploaderId());
            if (uploadAllowed) {
                if ((permission.isAddAttachment()) || (person.getId() == groupAccount.getUserId())) {
                    String identifier = new BigInteger(130, random).toString(32);
                    GroupAttachments attachment = new GroupAttachments(originalFile.getFileName(), new Timestamp(System.currentTimeMillis()),
                            this.personService.returnLoggedInUsersEmail(), person.getId(), originalFile.getFileSize(), groupAccount.getGroupId(), identifier,
                            msg + "groupattach/" + groupAccount.getGroupId() + "/", groupSection.getMsectionid(), groupCanvas.getMcanvasid(), groupNotes.getMnoticesid(), originalFile.getMimeType(), "");
                    attachment.setApiUsed("media");
                    if (attachmentUploader != null) {
                        attachment.setUploaderId(attachmentUploader.getId());
                    }
                    attachment.setFullSizePath(identifier);
                    if (originalFile.getFileUploadDate() != null) {
                        attachment.setFileUploadDate(originalFile.getFileUploadDate());
                    } else {
                        attachment.setFileUploadDate(new Timestamp(System.currentTimeMillis()));
                    }
                    final int[] attachId = {0};
                    attachId[0] = this.groupAttachmentsDAO.directAddAttachment(attachment, groupNotes);


                    try {
                        File file = new File(msg + "groupattach/" + groupAccount.getGroupId());
                        boolean creationResult = (file.exists() && file.isDirectory()) || new File(msg + "groupattach/" + groupAccount.getGroupId()).mkdir();
                        if (creationResult) {
                            OutputStream outputStream = new FileOutputStream(new File(msg + "groupattach/" + groupAccount.getGroupId() + "/" + identifier));
                            outputStream.write(originalFile.getSendAttachment());
                            outputStream.close();

                        }
                    } catch (Exception e) {
                     e.printStackTrace();

                    }

                    String extension = originalFile.getFileName().substring(originalFile.getFileName().lastIndexOf(".") + 1, originalFile.getFileName().length()).toLowerCase();

                    try {
                        GroupAttachments attachments = this.groupAttachmentsDAO.getAttachmenById(attachId[0]);
                        if (extension.toLowerCase().equals("zip")) {
                            Path path = Paths.get(msg + "groupattach/" + groupAccount.getGroupId() + "/" + identifier);
                            List<String> fileNames = getListOfFileNamesForZipFile(path);
                            String allNames = "";
                            int counter = 0;
                            for (String str : fileNames) {
                                allNames += str + ":/";
                                if (counter >= 50) {
                                    break;
                                }
                                counter++;
                            }
                            attachments.setFileNames(allNames);
                        }
                        if (extension.toLowerCase().equals("png") || (extension.equals("jpg")) || (extension.equals("jpeg"))) {
                            File outFileForWidth = new File(msg + "groupattach/" + groupAccount.getGroupId() + "/" + identifier);
                            BufferedImage bufferedImage = null;
                            try {
                                bufferedImage = ImageIO.read(outFileForWidth);
                            } catch (IOException e) {
                                e.printStackTrace();

                            }
                            if (bufferedImage != null) {
                                attachments.setWidth(bufferedImage.getWidth());
                                attachments.setHeight(bufferedImage.getHeight());
                                attachments.setNoteid(groupNotes.getMnoticesid());
                                this.groupAttachmentsDAO.updateAttachment(attachments);
                            }
                        }
                    } catch (Exception e) {
                     e.printStackTrace();

                    }

                    try {
                        GroupAttachments attachments = this.groupAttachmentsDAO.getAttachmenById(attachId[0]);
                        if (attachments != null) {
                            try {
                                GroupAttachments originAttachment = this.groupAttachmentsDAO.getAttachmenById(originalFile.getGroupAttachId());
                                if (originAttachment != null) {
                                    File originThumbnail = new File(msg + "thumbnail/" + originAttachment.getGroupId() + "/" + originAttachment.getFileIdentifier());
                                    if (originThumbnail.exists()) {
                                        if (originThumbnail.isFile()) {
                                            byte[] originThumbBytes = FileUtils.readFileToByteArray(originThumbnail);
                                            if (originThumbBytes != null) {
                                                File thumbnail = new File(msg + "thumbnail/" + groupAccount.getGroupId());
                                                boolean result = (thumbnail.exists() && thumbnail.isDirectory()) || new File(msg + "thumbnail/" + groupAccount.getGroupId()).mkdirs();
                                                if (result) {
                                                    OutputStream outputStream = new FileOutputStream(new File(msg + "thumbnail/" + groupAccount.getGroupId() + "/" + identifier));
                                                    outputStream.write(originThumbBytes);
                                                    outputStream.close();
                                                    attachments.setThumbNailAvailable(true);
                                                    this.groupAttachmentsDAO.updateAttachment(attachments);
                                                        addForNoteAndCanvasCopyForLucene(originalFile.getGroupAttachId(), attachId[0], false);

                                                }
                                            }
                                        }
                                    }
                                }

                            } catch (Exception e) {
                               e.printStackTrace();


                            }
                        }


                    } catch (Exception e) {
                       e.printStackTrace();

                    }

                    return true;
                }
            }
        }
    } catch (Exception e) {
        e.printStackTrace();

    }

    return false;
}

Update After adding some timings to the method, here is what I found out :

Start of method is 2018-04-11 13:17:24.212
Start of history entry is  2018-04-11 13:17:25.804
start of attachments is  2018-04-11 13:17:39.422
end of attachments is  2018-04-11 13:27:46.432

as you can see, the attachment part took 10 minutes to execute.

\$\endgroup\$
7
  • \$\begingroup\$ It'd be good to be able to relate those timings to points in code, e.g., where is Start of history entry... executed? Also, 10 minutes is a long range. Can't you add metrics inside the method addAttachmentForCopy()? \$\endgroup\$
    – antonro
    Apr 12, 2018 at 10:05
  • \$\begingroup\$ Did you find the problem? \$\endgroup\$
    – antonro
    Apr 23, 2018 at 7:55
  • \$\begingroup\$ @antonro : Sorry, Took me a long time to reply. the problem was with lucene indexing. There was a lot of data in it. cleaning up indexing mechanism and re-indexing did the trick. :-) \$\endgroup\$ Apr 23, 2018 at 8:43
  • \$\begingroup\$ So the call to addForNoteAndCanvasCopyForLucene(originalFile.getGroupAttachId(), attachId[0], false) ? \$\endgroup\$
    – antonro
    Apr 23, 2018 at 10:18
  • \$\begingroup\$ @antonro : Exactly. I had to reindex the data. For some reason, I was getting a lot of unprocessed data for storage in index. \$\endgroup\$ Apr 23, 2018 at 10:54

1 Answer 1

5
\$\begingroup\$

You said that the method addAttachmentForCopy() is taking a lot of time. This method is doing some DAO operations, but also a few file reading operations. Are you sure it's the DB access what's slowing it down? I wouldn't be surprised if bufferedImage = ImageIO.read(outFileForWidth); would take some time. What is the method getListOfFileNamesForZipFile(path) doing?

Not performance related, but logic like

String extension = originalFile.getFileName().substring(originalFile.getFileName().lastIndexOf(".") + 1, originalFile.getFileName().length()).toLowerCase();

very likely belongs in a getExtension() method in GroupAttachments.

Also, simplify things like if (!(originalFile.getSendAttachment() == null)) to if (originalFile.getSendAttachment() != null). You have also extra parenthesis around if ((permission.isAddAttachment()).

\$\endgroup\$
4
  • \$\begingroup\$ getListOfFilesNamesForZipFiles gives name of all the files present inside the zip. This information is saved in DB and shown in the zip preview. It was just a single line, so kept it in the method. Anything more you can see which is taking 5 minutes on the server. \$\endgroup\$ Apr 11, 2018 at 9:08
  • \$\begingroup\$ It's hard to tell. If the whole execution is taking 5 minutes, how long is the method addAttachmentForCopy() taking? Is there any way we can reproduce this, like a github repo? \$\endgroup\$
    – antonro
    Apr 11, 2018 at 9:24
  • \$\begingroup\$ I will add timers and let you know. Just give me around an hour... Thanks. \$\endgroup\$ Apr 11, 2018 at 9:30
  • \$\begingroup\$ I have added at bottom of main post the timings. Can you please check. Thank you. \$\endgroup\$ Apr 11, 2018 at 11:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.