JSON parse error when deleting "\n" character or pressing undo

:information_source: Attention Topic was automatically imported from the old Question2Answer platform.
:bust_in_silhouette: Asked By nadiavi

Hi, I’ve been working on a note app where I have a TextEdit where you can write a note and it will be save in JSON as a part of a dictionary entry along with some other info about the note.
I don’t have any issues with the saving and loading process and the json file is updated when TextEdit sends a signal when a change to the text is made. The error occurs when I try to delete a new line which is recorded in json as “\n”, if I press “Command z” for “undo”, or if I select a text and press a Backspace.
After that the json gets 2 additional }} characters at the end of the file and sometimes adds a part of the TextEdit text there as well and after that the parsing stops working. Also when I delete text in TextEdit - each character adds an empty line in the json file after the dictionary and when I type any text again the empty lines at the end get removed one by one.
I tried saving only when I click “Back” button, using text without strip_edges() and nothing worked.

Here is a screen recording of how the json changes when I type: youtube video
Here is my code in the note scene where I type text and save it:

extends Control

@onready var title = $VBoxContainer/Note/ScrollContainer/HBoxContainer/Title_and_text/title
@onready var content = $VBoxContainer/Note/ScrollContainer/HBoxContainer/Title_and_text/content

var bg = ""
var id = ""
var months = ["Jan", "Feb", "Mar", "Apr", "May", "June", "July", "Aug", "Sep", "Oct", "Nov", "Dec"]


#when new note opens clear the title and content 
func reset():
	title.text = ""
	content.text = ""
	bg = Global.COLORS.CARDS.pick_random()
	title.grab_focus()
	content.context_menu_enabled = false
	title.context_menu_enabled  = false
	gen_id()

#geretate id for each note using random letters
func gen_id():
	var chars = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"
	var id_length = 12
	id = ""
	for i in id_length:
		id += chars[randi()%chars.length()]

func _on_back_button_pressed() -> void:
	Global.go_back()

 
func save_note():
	var dates = Time.get_date_dict_from_system()
	var time = Time.get_unix_time_from_system()
	
	var note = {
		"title": title.text.strip_edges(),
		"content": content.text.strip_edges(),
		"bg": bg,
		"id": id,
		"date": "{day} {month}, {year}".format({
			"day": dates["day"],
			"month": months[dates["month"] - 1],
			"year": dates["year"]
		}), 
		"time_OS": int(time)
	}
	
	var json = JSON.new()
	#open the file 
	if not FileAccess.file_exists(Global.user_path):
		var file = FileAccess.open(Global.user_path, FileAccess.WRITE)
		var empty = {}
		var empty_string = JSON.stringify(empty, "\t")
		file.store_line(empty_string)
		file.close()
	else:
		var file = FileAccess.open(Global.user_path, FileAccess.READ_WRITE)		
		var parse_result = json.parse(file.get_as_text())
		if  parse_result != OK:
			print("JSON Parse Error in save", json.get_error_message())
		else:
			var data = json.get_data()
			data[id] = note
			var json_string_new = JSON.stringify(data, "\t")
			print("new line saved", json_string_new)
			file.store_line(json_string_new)
			file.close()


func set_note_data(data):
	content.grab_focus()
	id = data.id
	bg = data.bg
	title.text = data.title
	content.text = data.content
	

func _on_content_text_changed() -> void:
	save_note()
	
#
func _on_title_text_changed(new_text: String) -> void:
	save_note()

There seem to be a number of questionable things going on in your save_note() method, but I suspect the main issue could be related to the fact that you’re using READ_WRITE mode to overwrite your file. Since that DOES NOT truncate any existing file contents, if the file originally contained MORE data than you’re saving now, some of the data at the end will still remain. Since you’re writing the note in its entirety, you probably want to ensure that you start that operation with a truncated (empty) file. So, WRITE or WRITE_READ.

jgodfrey | 2023-03-19 17:49

You are right! Thank you so much, everything works now. Here is how I changed the code. If you have any thought on how to avoid opening and closing the json I would be happy to know.



if not FileAccess.file_exists(Global.user_path):
		var file_if_empty = FileAccess.open(Global.user_path, FileAccess.WRITE)
		var empty = {}
		var empty_string = JSON.stringify(empty, "\t")
		file_if_empty.store_line(empty_string)
		file_if_empty.close()
	else:
		#read the file and see if it has the note 
		var file_to_read = FileAccess.open(Global.user_path, FileAccess.READ)
		var parse_result = json.parse(file_to_read.get_as_text())
		if  parse_result != OK:
			print("JSON Parse Error in save ", json.get_error_message())
		else:
			var data = json.get_data()
			if !data.has(id):
				data[id] = note
			else:
				data.erase(id)
				data[id] = note
			file_to_read.close()
		# rewrite the json to store new file
			var file_to_write = FileAccess.open(Global.user_path, FileAccess.WRITE)
			var json_string_new = JSON.stringify(data, "\t")
			file_to_write.store_line(json_string_new)
			file_to_write.close()

nadiavi | 2023-03-19 18:56

I will say this seems more complex than necessary, but maybe you have some needs here that I don’t understand.

Rather than loading, parsing, updating, saving the JSON on every change, why not do something like:

  • On app start, load the existing “saved notes” file to a Godot Dictionary
  • While edits are being made, just update the Dictionary
  • When needed (either at User request, a timer event, something else), save the current copy of the Dictionary to your storage file.

That way, saving is just that - saving. No need to load, parse, compare, update, and resave.

That’d certainly be way simpler, much more efficient, and cause much less disk I/O. But, again, maybe that doesn’t fit our needs here…

jgodfrey | 2023-03-19 20:18

Yes, this is too much work as I just needed the note to be saved if the user closes the app, and I changed it to notifications instead. Thank you so much for your help!

func _notification(what: int) -> void:
	if what == NOTIFICATION_WM_CLOSE_REQUEST:
		save_note()
	if what == NOTIFICATION_CRASH:
		save_note()

nadiavi | 2023-03-20 08:32

:bust_in_silhouette: Reply From: jgodfrey

Based on the above conversation, the problem was caused by opening the save file in READ_WRITE mode, which DOES NOT truncate any existing file content - potentially leaving bits of the original file at the end of the new file.