Upload .gitkeep files/empty directories when publishing

[vashworth]

Flutter is working to adopt Swift Package Manager, and has run into an issue where directories with only a .gitkeep are not downloaded to pub-cache.

For example, plugin video_player_avfoundation (version 2.6.0) has video_player_avfoundation-2.6.0/darwin/video_player_avfoundation/Sources/video_player_avfoundation_ios/include/.gitkeep. However, in .pub-cache, both the include directory and the .gitkeep file are not found. This is a problem for Swift Package Manager as it requires an include directory.

I would expect /path/to/.pub-cache/hosted/pub.dev/video_player_avfoundation-2.6.0/darwin/video_player_avfoundation/Sources/video_player_avfoundation_ios/include/.gitkeep to exist

See flutter/flutter#148002

[jonasfj]

Just to clarify:

• Empty directories are not published.
• On publishing we ignore dot-files by default.

They are indeed extracted if present:

pub/lib/src/io.dart

Lines 1042 to 1138 in c2592b3

	Future<void> extractTarGz(Stream<List<int>> stream, String destination) async {
	log.fine('Extracting .tar.gz stream to $destination.');
	destination = p.absolute(destination);
	final reader = TarReader(stream.transform(gzip.decoder));
	final paths = <String>{};
	while (await reader.moveNext()) {
	final entry = reader.current;
	final filePath = p.joinAll([
	destination,
	// Tar file names always use forward slashes
	...p.posix.split(entry.name),
	]);
	if (!paths.add(filePath)) {
	// The tar file contained the same entry twice. Assume it is broken.
	await reader.cancel();
	throw FormatException('Tar file contained duplicate path ${entry.name}');
	}
	if (!p.isWithin(destination, filePath)) {
	// The tar contains entries that would be written outside of the
	// destination. That doesn't happen by accident, assume that the tar file
	// is malicious.
	await reader.cancel();
	throw FormatException('Invalid tar entry: ${entry.name}');
	}
	final parentDirectory = p.dirname(filePath);
	bool checkValidTarget(String linkTarget) {
	final isValid = p.isWithin(destination, linkTarget);
	if (!isValid) {
	log.fine('Skipping ${entry.name}: Invalid link target');
	}
	return isValid;
	}
	switch (entry.type) {
	case TypeFlag.dir:
	ensureDir(filePath);
	break;
	case TypeFlag.reg:
	case TypeFlag.regA:
	// Regular file
	deleteIfLink(filePath);
	ensureDir(parentDirectory);
	await createFileFromStream(entry.contents, filePath);
	if (Platform.isLinux || Platform.isMacOS) {
	// Apply executable bits from tar header, but don't change r/w bits
	// from the default
	final mode = _defaultMode | (entry.header.mode & _executableMask);
	if (mode != _defaultMode) {
	_chmod(mode, filePath);
	}
	}
	break;
	case TypeFlag.symlink:
	// Link to another file in this tar, relative from this entry.
	final resolvedTarget = p.joinAll(
	[parentDirectory, ...p.posix.split(entry.header.linkName!)],
	);
	if (!checkValidTarget(resolvedTarget)) {
	// Don't allow links to files outside of this tar.
	break;
	}
	ensureDir(parentDirectory);
	createSymlink(
	p.relative(resolvedTarget, from: parentDirectory),
	filePath,
	);
	break;
	case TypeFlag.link:
	// We generate hardlinks as symlinks too, but their linkName is relative
	// to the root of the tar file (unlike symlink entries, whose linkName
	// is relative to the entry itself).
	final fromDestination = p.join(destination, entry.header.linkName);
	if (!checkValidTarget(fromDestination)) {
	break; // Link points outside of the tar file.
	}
	final fromFile = p.relative(fromDestination, from: parentDirectory);
	ensureDir(parentDirectory);
	createSymlink(fromFile, filePath);
	break;
	default:
	// Only extract files
	continue;
	}
	}
	log.fine('Extracted .tar.gz to $destination.');
	}

[jonasfj]

Options for solving this:

• Support publishing empty directory.
• Support .gitkeep files, meaning do not ignore such files.
• Ask package authors to modify .pubignore to override the pattern ignoring .gitkeep.
• Ask package authors to include a README.md in empty directories instead.

@vashworth which of these would work?

[sigurdm]

The current workaround is to add a .pubignore next to the .gitkeep file, containing

!.gitkeep

[vashworth]

Options for solving this:

• Support publishing empty directory.
• Support .gitkeep files, meaning do not ignore such files.
• Ask package authors to modify .pubignore to override the pattern ignoring .gitkeep.
• Ask package authors to include a README.md in empty directories instead.

@vashworth which of these would work?

We decided as a workaround to use !.gitkeep to unblock us for now, but in my opinion, I think we should support publishing empty directories. Using .gitkeep to ensure an empty directory is tracked is pretty common practice, but it seems that people use it in different ways (https://github.com/search?q=path%3A.gitkeep&type=code)

[sigurdm]

Let us include empty directories in the published tar-file.